-- Logs begin at Thu 2022-07-14 16:54:20 UTC, end at Thu 2022-07-14 18:26:17 UTC. -- Jul 14 17:36:30.064328 dv-cinder-849890-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 14 17:36:30.071547 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 14 17:36:30.074776 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 14 17:36:30 2022] *** Jul 14 17:36:30.074776 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 14 17:36:30.074776 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 14 17:36:30.074776 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: nodename: dv-cinder-849890-1-iscsi Jul 14 17:36:30.074776 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: machine: x86_64 Jul 14 17:36:30.074776 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: clock source: unix Jul 14 17:36:30.074776 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: pcre jit disabled Jul 14 17:36:30.074776 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: detected number of CPU cores: 4 Jul 14 17:36:30.074776 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: current working directory: / Jul 14 17:36:30.074776 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: detected binary path: /usr/bin/uwsgi-core Jul 14 17:36:30.074776 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: your processes number limit is 47680 Jul 14 17:36:30.074776 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: your memory page size is 4096 bytes Jul 14 17:36:30.074776 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: detected max file descriptor number: 2048 Jul 14 17:36:30.074776 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: lock engine: pthread robust mutexes Jul 14 17:36:30.077340 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: thunder lock: enabled Jul 14 17:36:30.077340 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 14 17:36:30.077340 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 14 17:36:30.119660 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: Python main interpreter initialized at 0x55d43c8479c0 Jul 14 17:36:30.119802 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: python threads support enabled Jul 14 17:36:30.119802 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: your server socket listen backlog is limited to 100 connections Jul 14 17:36:30.119802 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: your mercy for graceful operations on workers is 90 seconds Jul 14 17:36:30.120089 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: mapped 403077 bytes (393 KB) for 2 cores Jul 14 17:36:30.120166 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: *** Operational MODE: preforking *** Jul 14 17:36:30.120388 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: *** uWSGI is running in multiple interpreter mode *** Jul 14 17:36:30.120388 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: spawned uWSGI master process (pid: 91918) Jul 14 17:36:30.120590 dv-cinder-849890-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 14 17:36:30.121997 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: spawned uWSGI worker 1 (pid: 91925, cores: 1) Jul 14 17:36:30.121997 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: spawned uWSGI worker 2 (pid: 91926, cores: 1) Jul 14 17:36:30.121997 dv-cinder-849890-1-iscsi devstack@keystone.service[91918]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 14 17:36:35.548022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 17:36:35.548022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: from cryptography.utils import int_from_bytes Jul 14 17:36:35.548780 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 17:36:35.548780 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: from cryptography.utils import int_from_bytes Jul 14 17:36:35.549186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 17:36:35.549186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: from cryptography.utils import int_from_bytes Jul 14 17:36:35.549294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 17:36:35.549294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: from cryptography.utils import int_from_bytes Jul 14 17:36:35.597670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 17:36:35.597829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 17:36:35.597913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 17:36:35.598004 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 17:36:35.598084 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] command line args: [] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 17:36:35.598168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] command line args: [] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 17:36:35.598299 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 17:36:35.598380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 17:36:35.598483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ================================================================================ {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 17:36:35.598564 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ================================================================================ {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 17:36:35.598742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] admin_token = **** {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.598832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] admin_token = **** {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.598990 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] config_dir = [] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.599077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] config_dir = [] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.599183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.599269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.599368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] config_source = [] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.599459 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] config_source = [] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.599538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] debug = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.599610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] debug = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.599737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601099 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601166 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] insecure_debug = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601493 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601620 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] list_limit = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] list_limit = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601743 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] log_config_append = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601802 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] log_config_append = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.601978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] log_dir = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.602034 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] log_dir = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.602101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] log_file = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.602164 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] log_file = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.602227 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.602286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.602347 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.602405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.602472 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.602530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.602587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.602646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.602705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.602768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.602883 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.603015 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.603129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.603228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.603369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.603489 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.603605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.603710 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.603836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.603968 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.604596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.604868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.605080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.605232 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.605531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.605648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.605764 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.605889 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] notification_format = cadf {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.606011 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.606130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.606374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.606512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.64/identity {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.606647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] publish_errors = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.606767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] publish_errors = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.606883 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.607010 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.607135 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.607257 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.607412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.607535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.607650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.607783 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.607910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.608055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.608179 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] standard_threads = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.608296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] standard_threads = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.608479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.608597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] strict_password_check = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.608712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.608827 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.608942 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.609056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] use_eventlog = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.609184 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] use_journal = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.609300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] use_journal = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.609415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] use_json = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.609529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] use_json = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.609644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] use_stderr = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.609764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] use_stderr = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.609888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] use_syslog = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.610004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] use_syslog = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.610119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.610235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] watch_log_file = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 17:36:35.610350 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.610476 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.610605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.610722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.610911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.611060 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.611185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.611318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.611437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.611552 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.611666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.611780 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.611894 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.612069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.612195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.612311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.612424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.612537 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.612647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.612777 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.612892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.613006 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.613119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.613231 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.613351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.613474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.613589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.613703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.613816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.613929 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.614042 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.614233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.614351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.614456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.614559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.614673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.614792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.614919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.615040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.615154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.615267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.615380 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.615494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.615618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.615738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.615852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.616023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.616145 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.616259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.616380 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.616495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.616608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.616721 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.616833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.616951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.617073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.617187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.617300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.617413 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.617531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.617656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.617772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.618145 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.618259 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.618372 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.618484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.618605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.618719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.618832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.618944 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.619056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.619169 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.619289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.619403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.619517 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.619629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.619742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.620485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.620683 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.620808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.620928 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.621041 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.621160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.621280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.621402 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.621517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.621630 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.621744 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.621857 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.621976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.622098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.622219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.622339 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.622452 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.622565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.622684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.622812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.622926 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.623046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.623165 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.623269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.623365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.623467 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.623569 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.623666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.623760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.623856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.624009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.624125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.624225 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.624340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.624454 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.624558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.624669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.624794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.624900 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.625012 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.625126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.625239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.625352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.625473 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.625595 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.625711 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.625833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.625950 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.626072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.626199 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.626315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.626436 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.626436 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.626651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.626764 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.626868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.626979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.627104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.627104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.627297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.627297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.627513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.627634 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.627756 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.627871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.627996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.628112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.628236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.628351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.628472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.628587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.628701 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.628701 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.628913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.628913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.629139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.enabled = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.629252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.629355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.629472 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.629593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.629715 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.629836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.629957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.630068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.630182 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.630295 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.630417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.630531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.630645 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.630758 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.630881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.630996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.631121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.631250 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.631371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.631486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.631612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.631726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.631914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.632073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.632187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.632312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.632426 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.632552 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.632676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.632799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.632926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.633040 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.633153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.633273 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.633387 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.633500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.633622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.633737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.633851 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.633965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.634084 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.634199 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.634320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.634435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.634550 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.634663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.634776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.634905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.635034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.635150 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.635264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.635378 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.635492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.635606 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.635727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] auth.external = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.635842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.635970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.636086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] auth.external = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.636200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.636327 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] auth.mapped = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.636471 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.636598 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.636723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] auth.password = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.636838 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.636953 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] auth.token = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.637067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] auth.password = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.637181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.637303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] auth.token = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.637419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.637534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.637648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.637762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] catalog.caching = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.637876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.638001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.638117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.638237 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.638361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.638476 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.638592 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.638714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.638830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] credential.caching = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.638944 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.639059 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.639173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] credential.caching = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.639294 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.639417 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] credential.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.639533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.639648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.639771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.639886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.640048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.640176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.640293 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.640407 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.640522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.640637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.640751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.640877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.640994 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.641109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.641223 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.641338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.641453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.641576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.641693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.641808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.641929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.642045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.642160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.642282 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.642398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.642513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.642627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] federation.caching = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.642743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.642864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.642991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] federation.caching = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.643116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.643232 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.643352 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.643468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] federation.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.643583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.643706 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.643823 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.643952 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.644069 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.644183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.644298 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.644425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.644541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.644656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.644771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.644887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.645001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.645124 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.645239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.645354 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.645468 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity.caching = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.645582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.645696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.645886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity.caching = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.646010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.646132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.646248 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.646363 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.646486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.646603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.646718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.646832 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.646948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.647062 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.647188 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.647303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.647418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.647532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.647646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.647760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.647886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.647886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.648140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.648254 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.648379 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.648494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.648617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.648617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.648832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.648946 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.649061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.649175 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.649302 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.649419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.649534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.649648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.649826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.649944 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.650069 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.650069 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.650284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.650399 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.650513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.650627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.650749 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.650870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.650985 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.651099 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.651214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.651329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.651451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.651567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.651681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.651796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.651911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.652047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.652175 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.652291 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.652406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.652521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.652636 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.652750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.652874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.652990 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.653105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.653221 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.653335 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.653450 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.653572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.653688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.653804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.653918 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.654033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.654148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.654272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.654388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.654503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.password = **** {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.654617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.654739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.654857 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.654983 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.655099 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.655214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.655330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.655450 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.655566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.655689 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.655804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.655919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.656081 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.656196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.656310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.656438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.656555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.656669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.656783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.656897 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.657011 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.657133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.657250 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.657364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.657479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.657593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.657713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.657837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.657953 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.658068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.658182 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.658297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.658411 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.658535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.658651 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.658766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.658881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.658995 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.659117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.659232 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.659347 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.659461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.659577 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.659691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.659813 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.659948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.660074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.660191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.660306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.660420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.660543 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.660660 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.660778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.660893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.661008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.661122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.661244 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.661361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.661475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.661591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.661705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.661819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.661942 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.662058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.662173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.662288 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.662409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.662525 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.662648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.662764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.662879 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.663000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.663116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.663230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.663352 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.663468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.663583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.663698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.663813 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.663951 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.664080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.664196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.664311 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.664425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] policy.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.664539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.664653 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.664776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.664892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.665007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.665122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.665236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.665350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] receipt.caching = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.665473 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.665589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.665703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.665819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.665947 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.666062 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.666185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] resource.caching = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.666300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.666415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.666530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.667127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.667243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] resource.caching = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.667365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.667481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.667596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.667711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] resource.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.667825 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.668010 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.668140 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.668257 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.668371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.668486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.668601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.668716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] revoke.caching = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.668893 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.669010 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.669125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] role.caching = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.669125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] role.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.669339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.669453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.669576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] role.cache_time = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.669692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.669807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] role.caching = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.669922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.670038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] role.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.670152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.670274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] role.list_limit = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.670391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.670505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.670619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.670738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.670738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.670962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.671079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.671194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.671310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.671426 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.671426 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.671649 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.671764 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.671879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.672014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.672131 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.672131 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.672360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.672360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.672657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.672657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.672873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.672997 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.673113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.673228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.673343 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.673458 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.673572 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.673695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.673811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.673926 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.674041 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.674156 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.674271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.674394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.674510 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.674624 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.674738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.674853 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.674967 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.675089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.675205 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.675205 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.675420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.675535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.675649 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.675773 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.675895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.675895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.676141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.676256 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.676384 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.676500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.676615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.676615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.676829 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.676943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.677065 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.677181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] token.caching = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.677295 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.677410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.677524 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.677639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.677761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] token.cache_time = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.677877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.677992 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] token.caching = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.678107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.678221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.678342 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.678467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] token.provider = fernet {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.678589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.678705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.678819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.678933 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.679088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.679212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.679328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.679444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.679558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.679672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.679787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.679910 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.680080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.680195 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] trust.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.680309 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.680424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.680543 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.680670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.680786 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.680901 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.681015 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.681129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.681243 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.681365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.681365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.681583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.connection = **** {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.681707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.681831 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.681831 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.682047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.682162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.682870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.connection = **** {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.683003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.683128 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.683245 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.683360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.683475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.683590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.683714 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.683830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.683963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.684081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.684196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.684310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.684438 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.684554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.684671 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.684786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.684907 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.685022 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.685145 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.685261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.685376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.685491 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.685606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.685722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.685844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.685960 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.686075 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.686189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.686305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.686419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.686542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91926) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 17:36:35.686659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.686659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.686659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 17:36:35.686659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=91925) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 17:36:35.714733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.714961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.715189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.715275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.715508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.715740 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.723607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.724233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.752873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.753099 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.753447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.753682 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.894787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.895176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.944214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.944406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.944516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:35.944642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.009197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.009426 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.009516 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.009654 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.009758 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.009877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.009969 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.010096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.010179 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.010313 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.010397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.010554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.035471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.035625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.035797 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.035882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.036000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.036080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.036155 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.036295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.036381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.036521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.036605 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.036741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.036825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.036957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.048609 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.048738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.061526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.061644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.061795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.061881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.061958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.062033 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.062188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.062282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.062406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.062485 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.062610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.062693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.062827 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91925) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.062918 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=91926) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 17:36:36.080129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.080338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 17:36:36.080422 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.080546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 17:36:36.080900 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.081102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.081229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 17:36:36.081312 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 17:36:36.081386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.081562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.081653 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.081823 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.081905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.082077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.082159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.082330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.094698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.094852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.094959 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.095060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.095160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.095279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.095382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.095483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.103001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.103108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.106915 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.107007 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.107197 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.107283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.107482 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.107579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.107766 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.107849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.108073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.108157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.108342 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.108428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.108611 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.108694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.108882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.108963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.109156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.109273 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.109428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.109514 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.109697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.109792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.136682 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.136930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.138315 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.138568 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.138726 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.138860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.143902 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.144201 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.144333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.144473 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.144603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.144757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.152548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.152778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.152874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.153095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.157313 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.157599 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.157687 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.157882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.157988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.158190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.158279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.158440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.158573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.158713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.158800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.158983 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.159068 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.159250 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.159349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.159516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.178622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.178903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.179050 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.179329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.183675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.184025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.187378 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.187672 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.187755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.188012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.192893 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.193223 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.196564 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.196929 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.200312 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.200608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.200691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.200901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.200986 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.201192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.201277 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.201470 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.201555 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.201757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.201850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.202073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.221559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.221839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.222035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.222134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.222319 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.222404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.222589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.222855 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.230941 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.231509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.232702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.232981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.233258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.233532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.236278 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.236558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.236771 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.236855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.237053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.237137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.237339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.237443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.237627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.237918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.251122 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.251400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.251678 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.251762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.251987 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.252094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.252270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.252355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.252551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.252636 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.252823 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.252909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.253117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.253396 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.272832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.273398 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.276715 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.277348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.280110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.280735 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.281731 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.282372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.283507 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.283785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.284099 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.284185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.284444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.284736 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.291461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.292258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.293142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.293908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.296431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.296722 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.297009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.297227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.297311 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.297517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.297801 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.298080 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.306604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.306891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.307170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.307456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.307746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.308052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.315157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91926) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.315437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.315716 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.316007 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=91925) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 17:36:36.316094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.316280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.316363 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.316558 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.316642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.316830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.316914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.317101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.317193 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.317388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.317472 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.317663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.317805 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.317938 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.318022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.318236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.318328 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.318518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.318602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.318785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91926) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.318868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.319079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.319355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.319624 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=91925) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 17:36:36.357926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d43c8479c0 pid: 91926 (default app) Jul 14 17:36:36.358850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d43c8479c0 pid: 91925 (default app) Jul 14 17:36:36.361335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbfeb5-4900-44db-a2e3-4a2f7575865e None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:36.361632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbfeb5-4900-44db-a2e3-4a2f7575865e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:36.361916 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dbfeb5-4900-44db-a2e3-4a2f7575865e None None] PATH_INFO: `/v3` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:36.366134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1/1] 10.222.0.64 () {50 vars in 791 bytes} [Thu Jul 14 17:36:36 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:36:43.964228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ff1ce-0f8c-4a1a-b6c4-26851a7553e4 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:43.964801 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ff1ce-0f8c-4a1a-b6c4-26851a7553e4 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:43.964801 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ff1ce-0f8c-4a1a-b6c4-26851a7553e4 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:43.969167 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1/2] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:36:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:36:43.974583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0192ba-81c6-45eb-9058-fefed6ad0be7 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:43.974993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0192ba-81c6-45eb-9058-fefed6ad0be7 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:43.975141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0192ba-81c6-45eb-9058-fefed6ad0be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:44.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: INFO dbcounter [None req-de0192ba-81c6-45eb-9058-fefed6ad0be7 None None] Registered counter for database keystone Jul 14 17:36:44.066440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG oslo_db.sqlalchemy.engines [None req-de0192ba-81c6-45eb-9058-fefed6ad0be7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91926) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 17:36:44.072621 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writer thread running {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 17:36:44.256081 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG passlib.handlers.bcrypt [None req-de0192ba-81c6-45eb-9058-fefed6ad0be7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91926) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 17:36:44.256434 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG passlib.handlers.bcrypt [None req-de0192ba-81c6-45eb-9058-fefed6ad0be7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91926) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 17:36:44.312310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-de0192ba-81c6-45eb-9058-fefed6ad0be7 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:36:44.313935 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-de0192ba-81c6-45eb-9058-fefed6ad0be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:36:44.437040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 2/3] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:36:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 14 17:36:44.628338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ff1ce-0f8c-4a1a-b6c4-26851a7553e4 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:44.698544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: INFO dbcounter [None req-ca5ff1ce-0f8c-4a1a-b6c4-26851a7553e4 None None] Registered counter for database keystone Jul 14 17:36:44.718099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG oslo_db.sqlalchemy.engines [None req-ca5ff1ce-0f8c-4a1a-b6c4-26851a7553e4 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=91925) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 17:36:44.724207 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writer thread running {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 17:36:44.841710 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ff1ce-0f8c-4a1a-b6c4-26851a7553e4 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:44.845955 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab6da68-a5e6-4a7c-af4e-30535bfa3baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:44.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6da68-a5e6-4a7c-af4e-30535bfa3baf admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:44.846895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6da68-a5e6-4a7c-af4e-30535bfa3baf admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:44.847174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab6da68-a5e6-4a7c-af4e-30535bfa3baf admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:44.855233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6da68-a5e6-4a7c-af4e-30535bfa3baf admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:44.906827 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fab6da68-a5e6-4a7c-af4e-30535bfa3baf admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 17:36:44.907113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab6da68-a5e6-4a7c-af4e-30535bfa3baf admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:44.916642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 2/4] 10.222.0.64 () {58 vars in 1324 bytes} [Thu Jul 14 17:36:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 17:36:44.923064 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0192ba-81c6-45eb-9058-fefed6ad0be7 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:44.938006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0192ba-81c6-45eb-9058-fefed6ad0be7 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:44.942420 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c23e79-8500-4504-98c3-309ef5dc4157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:44.943541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c23e79-8500-4504-98c3-309ef5dc4157 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:44.943833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c23e79-8500-4504-98c3-309ef5dc4157 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:44.944148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c23e79-8500-4504-98c3-309ef5dc4157 admin admin] PATH_INFO: `/v3/projects/29b8bf26bc324367a4fe27e90fc2d6df` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:44.948891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c23e79-8500-4504-98c3-309ef5dc4157 admin admin] RBAC: Authorizing `identity:get_project(project_id=29b8bf26bc324367a4fe27e90fc2d6df)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:44.999286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: WARNING keystone.common.rbac_enforcer.enforcer [None req-78c23e79-8500-4504-98c3-309ef5dc4157 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 17:36:44.999825 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c23e79-8500-4504-98c3-309ef5dc4157 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:45.004234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 3/5] 10.222.0.64 () {56 vars in 1207 bytes} [Thu Jul 14 17:36:44 2022] GET /identity/v3/projects/29b8bf26bc324367a4fe27e90fc2d6df => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:36:46.979980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd286c-4195-4444-8df8-3dc91dea06fc None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:46.980513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd286c-4195-4444-8df8-3dc91dea06fc None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:46.980610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dd286c-4195-4444-8df8-3dc91dea06fc None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:46.984683 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 3/6] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:36:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:36:46.990026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdd3c5-d05a-4ce8-a4c3-e94e6fc92ef3 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:46.990405 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdd3c5-d05a-4ce8-a4c3-e94e6fc92ef3 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:46.990590 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdd3c5-d05a-4ce8-a4c3-e94e6fc92ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:47.010096 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-5dfdd3c5-d05a-4ce8-a4c3-e94e6fc92ef3 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:36:47.011591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5dfdd3c5-d05a-4ce8-a4c3-e94e6fc92ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:36:47.019283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 4/7] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:36:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 14 17:36:47.040458 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd286c-4195-4444-8df8-3dc91dea06fc None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:47.053339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dd286c-4195-4444-8df8-3dc91dea06fc None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:47.056860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39289198-dcdc-4961-a81b-7da4bfc99a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:47.058069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39289198-dcdc-4961-a81b-7da4bfc99a0c admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:47.058352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39289198-dcdc-4961-a81b-7da4bfc99a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:47.058628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39289198-dcdc-4961-a81b-7da4bfc99a0c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:47.066216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39289198-dcdc-4961-a81b-7da4bfc99a0c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:47.068008 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39289198-dcdc-4961-a81b-7da4bfc99a0c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:47.076651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 4/8] 10.222.0.64 () {58 vars in 1324 bytes} [Thu Jul 14 17:36:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 17:36:47.082025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfdd3c5-d05a-4ce8-a4c3-e94e6fc92ef3 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:47.091120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfdd3c5-d05a-4ce8-a4c3-e94e6fc92ef3 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:47.095591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b88b4-2f63-47f2-8d68-7421f486ab63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:47.096563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b88b4-2f63-47f2-8d68-7421f486ab63 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:47.096854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b88b4-2f63-47f2-8d68-7421f486ab63 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:47.097135 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b88b4-2f63-47f2-8d68-7421f486ab63 admin admin] PATH_INFO: `/v3/users/7d77dedfd6bd435d94f994cfcde5100a` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:47.098539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b88b4-2f63-47f2-8d68-7421f486ab63 admin admin] RBAC: Authorizing `identity:get_user(user_id=7d77dedfd6bd435d94f994cfcde5100a)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:47.100440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b88b4-2f63-47f2-8d68-7421f486ab63 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:47.102034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 5/9] 10.222.0.64 () {56 vars in 1198 bytes} [Thu Jul 14 17:36:47 2022] GET /identity/v3/users/7d77dedfd6bd435d94f994cfcde5100a => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:36:49.160617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1342fa-ad0c-44a3-bdea-649b73948c8f None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:49.161099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1342fa-ad0c-44a3-bdea-649b73948c8f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:49.161282 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1342fa-ad0c-44a3-bdea-649b73948c8f None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:49.162658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 5/10] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:36:49.167907 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc1e7e-f771-49fc-aea5-47e331044dc2 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:49.168341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc1e7e-f771-49fc-aea5-47e331044dc2 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:49.168508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc1e7e-f771-49fc-aea5-47e331044dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:49.184932 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-09cc1e7e-f771-49fc-aea5-47e331044dc2 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:36:49.186113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-09cc1e7e-f771-49fc-aea5-47e331044dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:36:49.210730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecb9d6-b07d-4ceb-9bac-c12f659d7ba8 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:49.211016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecb9d6-b07d-4ceb-9bac-c12f659d7ba8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:49.211294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ecb9d6-b07d-4ceb-9bac-c12f659d7ba8 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:49.212740 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 6/11] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:36:49.218348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69672455-b215-45da-afc8-f74e0b846ea4 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:49.218640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69672455-b215-45da-afc8-f74e0b846ea4 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:49.218916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69672455-b215-45da-afc8-f74e0b846ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:49.258122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG passlib.handlers.bcrypt [None req-69672455-b215-45da-afc8-f74e0b846ea4 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=91925) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 17:36:49.258533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG passlib.handlers.bcrypt [None req-69672455-b215-45da-afc8-f74e0b846ea4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=91925) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 17:36:49.267328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 6/12] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:36:49.290825 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc1e7e-f771-49fc-aea5-47e331044dc2 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:49.304715 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-69672455-b215-45da-afc8-f74e0b846ea4 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:36:49.306299 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-69672455-b215-45da-afc8-f74e0b846ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:36:49.313822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:49.396485 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc1e7e-f771-49fc-aea5-47e331044dc2 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:49.404659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 7/13] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:36:49.426588 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69672455-b215-45da-afc8-f74e0b846ea4 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:49.433679 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6750ec-cd59-4200-a1f7-5564d6d6e245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:49.434493 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6750ec-cd59-4200-a1f7-5564d6d6e245 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:49.434782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6750ec-cd59-4200-a1f7-5564d6d6e245 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:49.435068 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6750ec-cd59-4200-a1f7-5564d6d6e245 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:49.446186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6750ec-cd59-4200-a1f7-5564d6d6e245 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-5f6750ec-cd59-4200-a1f7-5564d6d6e245 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:36:49.457937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:36:49.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:36:49.459661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:36:49.459661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:36:49.459661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:36:49.459661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:36:49.459661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:36:49.459661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:36:49.459661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:36:49.459661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:36:49.459661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:36:49.459661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:36:49.459661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:36:49.459661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:36:49.459661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 7/14] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:36:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:36:49.465199 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6750ec-cd59-4200-a1f7-5564d6d6e245 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:49.537602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69672455-b215-45da-afc8-f74e0b846ea4 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:49.539872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:49.545697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:49.565506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6750ec-cd59-4200-a1f7-5564d6d6e245 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:49.576700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bd063-5fce-44ec-be61-b57796d9319c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:49.577596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bd063-5fce-44ec-be61-b57796d9319c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:49.577882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bd063-5fce-44ec-be61-b57796d9319c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:49.578160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2bd063-5fce-44ec-be61-b57796d9319c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:49.598683 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e48daa-de4c-4d2c-99a2-b337197327a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:49.599354 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e48daa-de4c-4d2c-99a2-b337197327a1 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:49.599640 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e48daa-de4c-4d2c-99a2-b337197327a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:49.599940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e48daa-de4c-4d2c-99a2-b337197327a1 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:49.600713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e48daa-de4c-4d2c-99a2-b337197327a1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:36:49.601023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e48daa-de4c-4d2c-99a2-b337197327a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:49.602425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e48daa-de4c-4d2c-99a2-b337197327a1 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:49.613955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 8/15] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:36:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:36:49.620259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e48daa-de4c-4d2c-99a2-b337197327a1 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:49.644511 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bd063-5fce-44ec-be61-b57796d9319c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:49.646039 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2bd063-5fce-44ec-be61-b57796d9319c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:49.719447 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e48daa-de4c-4d2c-99a2-b337197327a1 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:49.741582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 8/16] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:36:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 17:36:49.747966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bd063-5fce-44ec-be61-b57796d9319c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:49.750560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:49.753973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e671e8-8d67-42ef-b179-cfd28af1496b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:49.754940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e671e8-8d67-42ef-b179-cfd28af1496b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:49.755301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e671e8-8d67-42ef-b179-cfd28af1496b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:49.755580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e671e8-8d67-42ef-b179-cfd28af1496b None admin] PATH_INFO: `/v3/users/7d77dedfd6bd435d94f994cfcde5100a` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:49.756833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e671e8-8d67-42ef-b179-cfd28af1496b None admin] RBAC: Authorizing `identity:get_user(user_id=7d77dedfd6bd435d94f994cfcde5100a)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:49.758297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e671e8-8d67-42ef-b179-cfd28af1496b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:49.759944 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 9/17] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:36:49 2022] GET /identity/v3/users/7d77dedfd6bd435d94f994cfcde5100a => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:36:49.765196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e671e8-8d67-42ef-b179-cfd28af1496b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:49.779386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:49.845792 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2bd063-5fce-44ec-be61-b57796d9319c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:49.863765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e671e8-8d67-42ef-b179-cfd28af1496b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:49.881334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c3469-57f6-47a8-8ac0-61dd2b41b62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:49.882132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c3469-57f6-47a8-8ac0-61dd2b41b62c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:49.882419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c3469-57f6-47a8-8ac0-61dd2b41b62c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:49.882696 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810c3469-57f6-47a8-8ac0-61dd2b41b62c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:49.899071 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cfd31-683b-4fa8-9c73-212d03c5aa36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:49.899871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cfd31-683b-4fa8-9c73-212d03c5aa36 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:49.900187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cfd31-683b-4fa8-9c73-212d03c5aa36 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:49.900470 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75cfd31-683b-4fa8-9c73-212d03c5aa36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:49.901419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c3469-57f6-47a8-8ac0-61dd2b41b62c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:49.901658 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cfd31-683b-4fa8-9c73-212d03c5aa36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:49.902892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810c3469-57f6-47a8-8ac0-61dd2b41b62c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:49.903104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75cfd31-683b-4fa8-9c73-212d03c5aa36 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:49.904744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 10/18] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:36:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:36:49.910596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cfd31-683b-4fa8-9c73-212d03c5aa36 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-810c3469-57f6-47a8-8ac0-61dd2b41b62c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:36:49.928043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:36:49.928905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:36:49.929698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:36:49.929698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:36:49.929698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:36:49.929698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:36:49.929698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 9/19] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:36:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:36:49.936409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c3469-57f6-47a8-8ac0-61dd2b41b62c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:49.991382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:49.996591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:50.016017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75cfd31-683b-4fa8-9c73-212d03c5aa36 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:50.037065 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810c3469-57f6-47a8-8ac0-61dd2b41b62c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:50.049131 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faca07a-4123-4a95-8cac-fe2b2bb4b3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:50.049796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faca07a-4123-4a95-8cac-fe2b2bb4b3ca None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:50.050088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faca07a-4123-4a95-8cac-fe2b2bb4b3ca None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:50.050369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faca07a-4123-4a95-8cac-fe2b2bb4b3ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:50.051208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faca07a-4123-4a95-8cac-fe2b2bb4b3ca None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.domain.id=default, user.id=7d77dedfd6bd435d94f994cfcde5100a) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:36:50.051521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faca07a-4123-4a95-8cac-fe2b2bb4b3ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:50.052996 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faca07a-4123-4a95-8cac-fe2b2bb4b3ca None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:50.060956 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 11/20] 10.222.0.64 () {56 vars in 1322 bytes} [Thu Jul 14 17:36:49 2022] GET /identity/v3/role_assignments?user.id=7d77dedfd6bd435d94f994cfcde5100a&scope.domain.id=default&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:36:50.071151 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fe14f-fc63-4ddc-8560-bf9c6b68d7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:50.071839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fe14f-fc63-4ddc-8560-bf9c6b68d7a8 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:50.072184 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fe14f-fc63-4ddc-8560-bf9c6b68d7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:50.072460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fe14f-fc63-4ddc-8560-bf9c6b68d7a8 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:50.073960 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fe14f-fc63-4ddc-8560-bf9c6b68d7a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:36:50.074385 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fe14f-fc63-4ddc-8560-bf9c6b68d7a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:50.076580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fe14f-fc63-4ddc-8560-bf9c6b68d7a8 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:50.096896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 10/21] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:36:52.392981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51b4f5-7696-431a-93f5-5fb313bd1efc None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:52.393455 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51b4f5-7696-431a-93f5-5fb313bd1efc None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:52.393455 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51b4f5-7696-431a-93f5-5fb313bd1efc None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:52.394825 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 12/22] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:36:52.412732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e817c-741e-40c5-95cb-2fd4537c6246 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:52.413218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e817c-741e-40c5-95cb-2fd4537c6246 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:52.413312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e817c-741e-40c5-95cb-2fd4537c6246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:52.436910 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-be8e817c-741e-40c5-95cb-2fd4537c6246 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:36:52.438105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-be8e817c-741e-40c5-95cb-2fd4537c6246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:36:52.517756 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 11/23] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:36:52.560793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51b4f5-7696-431a-93f5-5fb313bd1efc None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:52.618093 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ef873-034d-4af4-aa7b-daee4323eaad None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:52.618990 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ef873-034d-4af4-aa7b-daee4323eaad None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:52.619473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ef873-034d-4af4-aa7b-daee4323eaad None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:52.621120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 12/24] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:36:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:36:52.646982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288c52a-acd1-4cf3-845f-b12f428e405c None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:52.647583 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288c52a-acd1-4cf3-845f-b12f428e405c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:52.648138 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288c52a-acd1-4cf3-845f-b12f428e405c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:52.684995 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-d288c52a-acd1-4cf3-845f-b12f428e405c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:36:52.686549 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d288c52a-acd1-4cf3-845f-b12f428e405c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:36:52.707186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51b4f5-7696-431a-93f5-5fb313bd1efc None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:52.743802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae45fd1-51da-4250-a418-c25fcd4e8c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:52.744626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae45fd1-51da-4250-a418-c25fcd4e8c0c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:52.744941 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae45fd1-51da-4250-a418-c25fcd4e8c0c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:52.745220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae45fd1-51da-4250-a418-c25fcd4e8c0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:52.756046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae45fd1-51da-4250-a418-c25fcd4e8c0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-0ae45fd1-51da-4250-a418-c25fcd4e8c0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:36:52.764297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:36:52.765173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:36:52.765929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:36:52.765929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:36:52.765929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:36:52.765929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:36:52.765929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:36:52.765929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:36:52.765929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:36:52.765929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:36:52.765929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:36:52.765929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:36:52.765929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:36:52.765929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:36:52.765929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 13/25] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:36:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:36:52.769266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51faf4-16e2-4ecd-89ae-9eefebc5a556 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:52.769266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51faf4-16e2-4ecd-89ae-9eefebc5a556 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:52.769266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af51faf4-16e2-4ecd-89ae-9eefebc5a556 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:52.769512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:52.769797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 14/26] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:36:52.775440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51faf4-16e2-4ecd-89ae-9eefebc5a556 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:52.783009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:52.805870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 13/27] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:36:52.807491 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119196a6-8aab-4952-90f4-63f37f96cf44 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:52.807976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119196a6-8aab-4952-90f4-63f37f96cf44 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:52.808403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119196a6-8aab-4952-90f4-63f37f96cf44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:52.844937 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-119196a6-8aab-4952-90f4-63f37f96cf44 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:36:52.846372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-119196a6-8aab-4952-90f4-63f37f96cf44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:36:52.892196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af51faf4-16e2-4ecd-89ae-9eefebc5a556 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:52.929484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8dff01-0fc9-457d-b0e3-66345dc58df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:52.930373 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8dff01-0fc9-457d-b0e3-66345dc58df6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:52.930797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8dff01-0fc9-457d-b0e3-66345dc58df6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:52.931211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8dff01-0fc9-457d-b0e3-66345dc58df6 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:52.932150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8dff01-0fc9-457d-b0e3-66345dc58df6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:36:52.933193 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8dff01-0fc9-457d-b0e3-66345dc58df6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:52.934701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8dff01-0fc9-457d-b0e3-66345dc58df6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:52.943299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 15/28] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:36:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:36:52.944425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8dff01-0fc9-457d-b0e3-66345dc58df6 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:52.953327 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 14/29] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:36:52.953999 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119196a6-8aab-4952-90f4-63f37f96cf44 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:53.069514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:53.129888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119196a6-8aab-4952-90f4-63f37f96cf44 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:53.139252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:53.149326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8dff01-0fc9-457d-b0e3-66345dc58df6 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:53.171377 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2efb8-6859-40d4-9bc5-396d4d65b7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:53.172560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2efb8-6859-40d4-9bc5-396d4d65b7a7 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:53.172991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2efb8-6859-40d4-9bc5-396d4d65b7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:53.173410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2efb8-6859-40d4-9bc5-396d4d65b7a7 None admin] PATH_INFO: `/v3/users/7d77dedfd6bd435d94f994cfcde5100a` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:53.175339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2efb8-6859-40d4-9bc5-396d4d65b7a7 None admin] RBAC: Authorizing `identity:get_user(user_id=7d77dedfd6bd435d94f994cfcde5100a)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:53.177001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2efb8-6859-40d4-9bc5-396d4d65b7a7 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:53.178812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 15/30] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:36:52 2022] GET /identity/v3/users/7d77dedfd6bd435d94f994cfcde5100a => generated 247 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:36:53.179896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2efb8-6859-40d4-9bc5-396d4d65b7a7 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:53.194268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc8a3f-1707-47de-8e38-58f96bddeee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:53.195100 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc8a3f-1707-47de-8e38-58f96bddeee6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:53.195654 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc8a3f-1707-47de-8e38-58f96bddeee6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:53.195962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc8a3f-1707-47de-8e38-58f96bddeee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:53.210475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc8a3f-1707-47de-8e38-58f96bddeee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:53.212606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebc8a3f-1707-47de-8e38-58f96bddeee6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-8ebc8a3f-1707-47de-8e38-58f96bddeee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:36:53.236892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:36:53.237752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:36:53.238540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:36:53.238540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:36:53.238540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:36:53.238540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:36:53.238540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 16/31] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:36:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:36:53.239766 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062036fc-b23d-420a-94d7-22115014319a None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:53.240113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062036fc-b23d-420a-94d7-22115014319a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:53.240685 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062036fc-b23d-420a-94d7-22115014319a None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:53.246437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 17/32] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:36:53.246437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062036fc-b23d-420a-94d7-22115014319a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:53.309611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2efb8-6859-40d4-9bc5-396d4d65b7a7 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:53.353004 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062036fc-b23d-420a-94d7-22115014319a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:53.353967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7364d8-d909-40f6-a047-4e32b0101557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:53.355227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7364d8-d909-40f6-a047-4e32b0101557 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:53.355517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7364d8-d909-40f6-a047-4e32b0101557 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:53.355800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7364d8-d909-40f6-a047-4e32b0101557 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:53.356883 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7364d8-d909-40f6-a047-4e32b0101557 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:53.359204 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7364d8-d909-40f6-a047-4e32b0101557 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:53.376713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:53.397102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa61636-ad14-4eb7-9334-ae77f8a5b4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:53.398890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa61636-ad14-4eb7-9334-ae77f8a5b4e9 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:53.399430 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa61636-ad14-4eb7-9334-ae77f8a5b4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:53.400083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa61636-ad14-4eb7-9334-ae77f8a5b4e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:53.402242 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa61636-ad14-4eb7-9334-ae77f8a5b4e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:53.403952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa61636-ad14-4eb7-9334-ae77f8a5b4e9 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:53.405943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 18/33] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:36:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:36:53.407219 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa61636-ad14-4eb7-9334-ae77f8a5b4e9 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:53.513374 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 16/34] 10.222.0.64 () {60 vars in 1131 bytes} [Thu Jul 14 17:36:53 2022] POST /identity/v3/roles => generated 219 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:36:53.515725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de53ee9-e16c-46c5-ba61-558a6cc9cf57 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:53.516294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de53ee9-e16c-46c5-ba61-558a6cc9cf57 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:53.516852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de53ee9-e16c-46c5-ba61-558a6cc9cf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:53.521424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa61636-ad14-4eb7-9334-ae77f8a5b4e9 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:53.537392 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-0de53ee9-e16c-46c5-ba61-558a6cc9cf57 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:36:53.539364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0de53ee9-e16c-46c5-ba61-558a6cc9cf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:36:53.566411 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989350c-19f8-4827-bb9c-a32a9659a8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:53.567078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989350c-19f8-4827-bb9c-a32a9659a8a0 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:53.567364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989350c-19f8-4827-bb9c-a32a9659a8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:53.567647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989350c-19f8-4827-bb9c-a32a9659a8a0 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:53.568605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989350c-19f8-4827-bb9c-a32a9659a8a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:36:53.568919 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989350c-19f8-4827-bb9c-a32a9659a8a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:53.571125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989350c-19f8-4827-bb9c-a32a9659a8a0 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:53.574152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:53.606531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 19/35] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:36:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:36:53.607412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989350c-19f8-4827-bb9c-a32a9659a8a0 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:53.663312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 17/36] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:36:53.664218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de53ee9-e16c-46c5-ba61-558a6cc9cf57 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:53.706337 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:53.774470 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989350c-19f8-4827-bb9c-a32a9659a8a0 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:53.812842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624ed96-8153-4c59-92b5-bda8782f7da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:53.814162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624ed96-8153-4c59-92b5-bda8782f7da8 None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:53.814646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624ed96-8153-4c59-92b5-bda8782f7da8 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:53.815124 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624ed96-8153-4c59-92b5-bda8782f7da8 None admin] PATH_INFO: `/v3/domains/default/users/7d77dedfd6bd435d94f994cfcde5100a/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:53.817396 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624ed96-8153-4c59-92b5-bda8782f7da8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7d77dedfd6bd435d94f994cfcde5100a, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:53.818434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de53ee9-e16c-46c5-ba61-558a6cc9cf57 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:53.819449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624ed96-8153-4c59-92b5-bda8782f7da8 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:53.862651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa2291-ca92-4515-a4d1-333dbd968230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:53.863331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa2291-ca92-4515-a4d1-333dbd968230 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:53.863623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa2291-ca92-4515-a4d1-333dbd968230 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:53.863904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa2291-ca92-4515-a4d1-333dbd968230 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:53.864686 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa2291-ca92-4515-a4d1-333dbd968230 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:53.866273 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa2291-ca92-4515-a4d1-333dbd968230 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:53.897841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 18/37] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Jul 14 17:36:53 2022] POST /identity/v3/projects => generated 301 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:36:53.898748 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa2291-ca92-4515-a4d1-333dbd968230 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:53.927059 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:53.949290 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 20/38] 10.222.0.64 () {58 vars in 1361 bytes} [Thu Jul 14 17:36:53 2022] PUT /identity/v3/domains/default/users/7d77dedfd6bd435d94f994cfcde5100a/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:36:53.951513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b337b7c-5d02-4b3c-a02d-cd3f5a71b805 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:53.951824 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b337b7c-5d02-4b3c-a02d-cd3f5a71b805 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:53.952250 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b337b7c-5d02-4b3c-a02d-cd3f5a71b805 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:53.957276 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 21/39] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:36:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:36:53.963483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d55e9-2afc-4acf-8047-17919130f0e6 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:53.964483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d55e9-2afc-4acf-8047-17919130f0e6 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:53.964882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d55e9-2afc-4acf-8047-17919130f0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:53.985015 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-4e8d55e9-2afc-4acf-8047-17919130f0e6 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:36:53.986931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4e8d55e9-2afc-4acf-8047-17919130f0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:36:54.043578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa2291-ca92-4515-a4d1-333dbd968230 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:54.088436 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 22/40] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:36:54.111061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e71be-978b-466d-801e-897aa4caad40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:54.111763 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e71be-978b-466d-801e-897aa4caad40 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:54.112097 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e71be-978b-466d-801e-897aa4caad40 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:54.112383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e71be-978b-466d-801e-897aa4caad40 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:54.113163 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e71be-978b-466d-801e-897aa4caad40 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:54.114608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420e71be-978b-466d-801e-897aa4caad40 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:54.115336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d55e9-2afc-4acf-8047-17919130f0e6 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:54.139079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 19/41] 10.222.0.64 () {60 vars in 1131 bytes} [Thu Jul 14 17:36:53 2022] POST /identity/v3/roles => generated 225 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:36:54.170458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:54.186670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd8da7-5731-40a7-bda6-da7b72c1426c None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:54.186969 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd8da7-5731-40a7-bda6-da7b72c1426c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:54.187261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd8da7-5731-40a7-bda6-da7b72c1426c None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:54.189773 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 20/42] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:36:54.195401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37168661-d615-4544-bb28-8e11dfa51396 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:54.195746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37168661-d615-4544-bb28-8e11dfa51396 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:54.196068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37168661-d615-4544-bb28-8e11dfa51396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:54.226205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-37168661-d615-4544-bb28-8e11dfa51396 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:36:54.227629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-37168661-d615-4544-bb28-8e11dfa51396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:36:54.265943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8d55e9-2afc-4acf-8047-17919130f0e6 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:54.319808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 21/43] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:36:54.343833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37168661-d615-4544-bb28-8e11dfa51396 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:54.358360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:54.413812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99318308-e8c3-4b15-af4c-9a66a0d18efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:54.414501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99318308-e8c3-4b15-af4c-9a66a0d18efd None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:54.414773 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99318308-e8c3-4b15-af4c-9a66a0d18efd None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:54.415055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99318308-e8c3-4b15-af4c-9a66a0d18efd None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:54.415897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99318308-e8c3-4b15-af4c-9a66a0d18efd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:54.417891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99318308-e8c3-4b15-af4c-9a66a0d18efd None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:54.433557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 23/44] 10.222.0.64 () {60 vars in 1131 bytes} [Thu Jul 14 17:36:54 2022] POST /identity/v3/roles => generated 223 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:36:54.459506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37168661-d615-4544-bb28-8e11dfa51396 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:54.499294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aca89b-322f-4542-86cd-7d04753a1f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:54.500142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aca89b-322f-4542-86cd-7d04753a1f7a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:54.500429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aca89b-322f-4542-86cd-7d04753a1f7a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:54.500708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aca89b-322f-4542-86cd-7d04753a1f7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:54.501976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aca89b-322f-4542-86cd-7d04753a1f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:54.503489 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aca89b-322f-4542-86cd-7d04753a1f7a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:54.505282 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 22/45] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:36:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:36:54.510965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99318308-e8c3-4b15-af4c-9a66a0d18efd None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:54.616214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:54.624199 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99318308-e8c3-4b15-af4c-9a66a0d18efd None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:54.660574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c85e0-0150-4375-8b72-f6e6e1c7e398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:54.661246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c85e0-0150-4375-8b72-f6e6e1c7e398 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:54.661537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c85e0-0150-4375-8b72-f6e6e1c7e398 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:54.661821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c85e0-0150-4375-8b72-f6e6e1c7e398 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:54.662539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c85e0-0150-4375-8b72-f6e6e1c7e398 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:54.664046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c85e0-0150-4375-8b72-f6e6e1c7e398 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:54.690695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 24/46] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Jul 14 17:36:54 2022] POST /identity/v3/projects => generated 312 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:36:56.107745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb79baf-7923-4c59-8e3c-45b09075f0fd None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:56.108289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb79baf-7923-4c59-8e3c-45b09075f0fd None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:56.108372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb79baf-7923-4c59-8e3c-45b09075f0fd None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:56.109725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 23/47] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:36:56.114962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af40fe-b2f6-49c9-85b2-4bd7a4370ca3 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:56.115347 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af40fe-b2f6-49c9-85b2-4bd7a4370ca3 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:56.115545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af40fe-b2f6-49c9-85b2-4bd7a4370ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:56.133266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-63af40fe-b2f6-49c9-85b2-4bd7a4370ca3 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:36:56.134481 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-63af40fe-b2f6-49c9-85b2-4bd7a4370ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:36:56.200223 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 25/48] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:36:56.224273 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb79baf-7923-4c59-8e3c-45b09075f0fd None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:56.298741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:56.323426 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb79baf-7923-4c59-8e3c-45b09075f0fd None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:56.356835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38319b8f-9917-4606-b724-b3c8f36ef47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:56.357636 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38319b8f-9917-4606-b724-b3c8f36ef47c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:56.357972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38319b8f-9917-4606-b724-b3c8f36ef47c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:56.358203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38319b8f-9917-4606-b724-b3c8f36ef47c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:56.369355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38319b8f-9917-4606-b724-b3c8f36ef47c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-38319b8f-9917-4606-b724-b3c8f36ef47c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:36:56.378114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:36:56.379019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:36:56.379784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:36:56.379784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:36:56.379784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:36:56.379784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:36:56.379784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:36:56.379784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:36:56.379784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:36:56.379784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:36:56.379784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:36:56.379784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:36:56.379784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:36:56.379784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:36:56.379784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 24/49] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:36:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:36:56.385052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af40fe-b2f6-49c9-85b2-4bd7a4370ca3 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:56.459829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:56.482763 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af40fe-b2f6-49c9-85b2-4bd7a4370ca3 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:56.516124 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729d9e5-25e8-43d5-96bd-5e4b075dabec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:56.516796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729d9e5-25e8-43d5-96bd-5e4b075dabec None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:56.517085 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729d9e5-25e8-43d5-96bd-5e4b075dabec None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:56.517366 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729d9e5-25e8-43d5-96bd-5e4b075dabec None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:56.518119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729d9e5-25e8-43d5-96bd-5e4b075dabec None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:36:56.518430 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729d9e5-25e8-43d5-96bd-5e4b075dabec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:56.519817 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a729d9e5-25e8-43d5-96bd-5e4b075dabec None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:56.528458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 26/50] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:36:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:36:56.534115 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38319b8f-9917-4606-b724-b3c8f36ef47c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:56.630552 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38319b8f-9917-4606-b724-b3c8f36ef47c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:56.663404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e59103-3a70-418e-8dab-f5fada087355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:56.664378 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e59103-3a70-418e-8dab-f5fada087355 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:56.664667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e59103-3a70-418e-8dab-f5fada087355 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:56.664946 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e59103-3a70-418e-8dab-f5fada087355 None admin] PATH_INFO: `/v3/users/7d77dedfd6bd435d94f994cfcde5100a` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:56.666132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e59103-3a70-418e-8dab-f5fada087355 None admin] RBAC: Authorizing `identity:get_user(user_id=7d77dedfd6bd435d94f994cfcde5100a)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:56.667631 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e59103-3a70-418e-8dab-f5fada087355 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:56.669289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 25/51] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:36:56 2022] GET /identity/v3/users/7d77dedfd6bd435d94f994cfcde5100a => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:36:56.674798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729d9e5-25e8-43d5-96bd-5e4b075dabec None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:56.727344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51669675-67d3-4426-8ab3-feeb7ece19f7 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:56.727652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51669675-67d3-4426-8ab3-feeb7ece19f7 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:56.728206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51669675-67d3-4426-8ab3-feeb7ece19f7 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:56.729434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 26/52] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:36:56.734846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc29a7-3fb5-4304-b980-ff757ea11ff8 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:56.735191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc29a7-3fb5-4304-b980-ff757ea11ff8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:56.735536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadc29a7-3fb5-4304-b980-ff757ea11ff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:56.740630 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:56.753071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-dadc29a7-3fb5-4304-b980-ff757ea11ff8 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:36:56.754262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-dadc29a7-3fb5-4304-b980-ff757ea11ff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:36:56.780339 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729d9e5-25e8-43d5-96bd-5e4b075dabec None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:56.813103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171eb4f-57de-4a75-85f8-c328a2000e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:56.813920 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171eb4f-57de-4a75-85f8-c328a2000e50 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:56.814212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171eb4f-57de-4a75-85f8-c328a2000e50 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:56.814496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171eb4f-57de-4a75-85f8-c328a2000e50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:56.815666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171eb4f-57de-4a75-85f8-c328a2000e50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:56.817177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171eb4f-57de-4a75-85f8-c328a2000e50 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:56.818799 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 27/53] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:36:56.819285 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 27/54] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:36:56.824957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171eb4f-57de-4a75-85f8-c328a2000e50 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:56.835524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:56.843104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc29a7-3fb5-4304-b980-ff757ea11ff8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:56.923185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171eb4f-57de-4a75-85f8-c328a2000e50 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:56.939949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadc29a7-3fb5-4304-b980-ff757ea11ff8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:56.956623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a95438-be6a-4bc3-82f2-7f25e2382d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:56.957287 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a95438-be6a-4bc3-82f2-7f25e2382d08 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:56.957570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a95438-be6a-4bc3-82f2-7f25e2382d08 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:56.957858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a95438-be6a-4bc3-82f2-7f25e2382d08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:56.958690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a95438-be6a-4bc3-82f2-7f25e2382d08 None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.domain.id=default, user.id=7d77dedfd6bd435d94f994cfcde5100a) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:36:56.959008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a95438-be6a-4bc3-82f2-7f25e2382d08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:56.960499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a95438-be6a-4bc3-82f2-7f25e2382d08 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:56.966269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 28/55] 10.222.0.64 () {56 vars in 1322 bytes} [Thu Jul 14 17:36:56 2022] GET /identity/v3/role_assignments?user.id=7d77dedfd6bd435d94f994cfcde5100a&scope.domain.id=default&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:36:56.969493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:56.975473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9709e-1339-4e50-a4d0-20556e532626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:56.976331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9709e-1339-4e50-a4d0-20556e532626 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:56.976638 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9709e-1339-4e50-a4d0-20556e532626 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:56.976904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9709e-1339-4e50-a4d0-20556e532626 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:56.978194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9709e-1339-4e50-a4d0-20556e532626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:56.979746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9709e-1339-4e50-a4d0-20556e532626 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:56.981512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 28/56] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:36:56.987406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a95438-be6a-4bc3-82f2-7f25e2382d08 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:57.062985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:57.094301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a95438-be6a-4bc3-82f2-7f25e2382d08 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:57.130874 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c66b97f-9307-4218-98c6-078efd2a05a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:57.131884 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66b97f-9307-4218-98c6-078efd2a05a7 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:57.131884 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66b97f-9307-4218-98c6-078efd2a05a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:57.132132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c66b97f-9307-4218-98c6-078efd2a05a7 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:57.132862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66b97f-9307-4218-98c6-078efd2a05a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:57.134304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c66b97f-9307-4218-98c6-078efd2a05a7 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:57.160309 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 29/57] 10.222.0.64 () {60 vars in 1140 bytes} [Thu Jul 14 17:36:56 2022] POST /identity/v3/projects => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:36:59.176532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c308ba-1228-4b54-94b0-af572568dfd8 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:59.177004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c308ba-1228-4b54-94b0-af572568dfd8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:59.177118 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c308ba-1228-4b54-94b0-af572568dfd8 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:59.178462 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 29/58] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:36:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:36:59.184057 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf4efe-76e1-4bca-a129-3b3b6d10fdbd None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:59.184421 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf4efe-76e1-4bca-a129-3b3b6d10fdbd None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:59.184630 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf4efe-76e1-4bca-a129-3b3b6d10fdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:59.201413 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-a9bf4efe-76e1-4bca-a129-3b3b6d10fdbd None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:36:59.202614 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a9bf4efe-76e1-4bca-a129-3b3b6d10fdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:36:59.267103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 30/59] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:36:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:36:59.290952 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c308ba-1228-4b54-94b0-af572568dfd8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:59.491832 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c308ba-1228-4b54-94b0-af572568dfd8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:59.524368 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5e5de-3951-4d42-88fc-8fa409daf46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:59.525140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5e5de-3951-4d42-88fc-8fa409daf46f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:59.525422 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5e5de-3951-4d42-88fc-8fa409daf46f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:59.525698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5e5de-3951-4d42-88fc-8fa409daf46f None admin] PATH_INFO: `/v3/domains/default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:59.526886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5e5de-3951-4d42-88fc-8fa409daf46f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:59.528344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5e5de-3951-4d42-88fc-8fa409daf46f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:59.529932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 30/60] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:36:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:36:59.535703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf4efe-76e1-4bca-a129-3b3b6d10fdbd None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:36:59.588744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:36:59.632424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf4efe-76e1-4bca-a129-3b3b6d10fdbd None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:36:59.664378 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab0d47-05d6-4f0c-8676-583edcbba7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:36:59.665046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab0d47-05d6-4f0c-8676-583edcbba7fc None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:36:59.665332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab0d47-05d6-4f0c-8676-583edcbba7fc None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:36:59.665611 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab0d47-05d6-4f0c-8676-583edcbba7fc None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:36:59.666312 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab0d47-05d6-4f0c-8676-583edcbba7fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:36:59.668302 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab0d47-05d6-4f0c-8676-583edcbba7fc None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:36:59.694728 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 31/61] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:36:59 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:37:02.910252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b99765-e8fe-4bd9-a398-36c58ea30922 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:02.911286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b99765-e8fe-4bd9-a398-36c58ea30922 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:02.911779 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b99765-e8fe-4bd9-a398-36c58ea30922 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:02.913406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 31/62] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:02.918898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906eeb42-bfe6-45fc-a082-a3eb1317815f None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:02.919321 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906eeb42-bfe6-45fc-a082-a3eb1317815f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:02.919465 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906eeb42-bfe6-45fc-a082-a3eb1317815f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:02.940051 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-906eeb42-bfe6-45fc-a082-a3eb1317815f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:02.941284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-906eeb42-bfe6-45fc-a082-a3eb1317815f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:03.036437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 32/63] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:03.060427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b99765-e8fe-4bd9-a398-36c58ea30922 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:03.073947 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bb4a6-cb72-4e58-9eaa-6de76a09e97f None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.074488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bb4a6-cb72-4e58-9eaa-6de76a09e97f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.074915 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bb4a6-cb72-4e58-9eaa-6de76a09e97f None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.076706 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 33/64] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:03.081890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197c0a1-7db9-4116-8188-853db90a1c30 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.082348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197c0a1-7db9-4116-8188-853db90a1c30 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.082797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8197c0a1-7db9-4116-8188-853db90a1c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.115037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-8197c0a1-7db9-4116-8188-853db90a1c30 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:03.116595 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8197c0a1-7db9-4116-8188-853db90a1c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:03.166978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:03.185943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:03.231099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b99765-e8fe-4bd9-a398-36c58ea30922 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:03.237256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 34/65] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:03.238613 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970b417-8057-49d7-9ef6-d5cddbf90b03 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.238908 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970b417-8057-49d7-9ef6-d5cddbf90b03 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.240094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a970b417-8057-49d7-9ef6-d5cddbf90b03 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.241471 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 35/66] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:03.256758 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd037950-761a-4d7a-a7c8-a8c2c1eae693 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.257052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd037950-761a-4d7a-a7c8-a8c2c1eae693 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.257339 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd037950-761a-4d7a-a7c8-a8c2c1eae693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.276873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-bd037950-761a-4d7a-a7c8-a8c2c1eae693 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:03.278075 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-bd037950-761a-4d7a-a7c8-a8c2c1eae693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:03.285181 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67bcc-c12d-43d9-8d4a-f7a3d691f608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:03.286094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67bcc-c12d-43d9-8d4a-f7a3d691f608 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.286382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67bcc-c12d-43d9-8d4a-f7a3d691f608 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.286659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67bcc-c12d-43d9-8d4a-f7a3d691f608 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.298024 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db67bcc-c12d-43d9-8d4a-f7a3d691f608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-8db67bcc-c12d-43d9-8d4a-f7a3d691f608 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:03.306258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:03.307139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:03.308505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:03.308505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:03.308505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:03.308505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:03.308505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:03.308505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:03.308505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:03.308505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:03.308505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:03.308505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:03.308505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:03.308505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:03.308505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 32/67] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:03.309595 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad4052-d3db-48d9-bd08-5bf1225649ac None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.309595 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad4052-d3db-48d9-bd08-5bf1225649ac None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.309941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad4052-d3db-48d9-bd08-5bf1225649ac None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.311173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 33/68] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:03.311878 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad4052-d3db-48d9-bd08-5bf1225649ac None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:03.359144 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 36/69] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:03.360133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd037950-761a-4d7a-a7c8-a8c2c1eae693 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:03.416276 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad4052-d3db-48d9-bd08-5bf1225649ac None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:03.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:03.450940 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8be79-7776-4d2c-be1a-23687b1830ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:03.451732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8be79-7776-4d2c-be1a-23687b1830ef None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.452048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8be79-7776-4d2c-be1a-23687b1830ef None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.452333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8be79-7776-4d2c-be1a-23687b1830ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.455639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:03.458340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd037950-761a-4d7a-a7c8-a8c2c1eae693 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:03.462745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8be79-7776-4d2c-be1a-23687b1830ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-3ff8be79-7776-4d2c-be1a-23687b1830ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:03.470434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:03.471301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:03.472104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:03.472104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:03.472104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:03.472104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:03.472104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:03.472104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:03.472104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:03.472104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:03.472104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:03.472104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:03.472104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:03.472104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:03.472104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 34/70] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:37:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:03.473029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d768ebe-ee95-4800-9805-20d1a48036ba None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.473328 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d768ebe-ee95-4800-9805-20d1a48036ba None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.473618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d768ebe-ee95-4800-9805-20d1a48036ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.490616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5d768ebe-ee95-4800-9805-20d1a48036ba None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:03.492047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5d768ebe-ee95-4800-9805-20d1a48036ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:03.501326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40afa2b-55a8-47f9-8c15-7914b3a6f3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:03.502008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40afa2b-55a8-47f9-8c15-7914b3a6f3dc None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.502299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40afa2b-55a8-47f9-8c15-7914b3a6f3dc None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.502585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40afa2b-55a8-47f9-8c15-7914b3a6f3dc None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.503333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40afa2b-55a8-47f9-8c15-7914b3a6f3dc None admin] RBAC: Adding query filter params (name=member) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:03.503646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40afa2b-55a8-47f9-8c15-7914b3a6f3dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:03.505066 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40afa2b-55a8-47f9-8c15-7914b3a6f3dc None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:03.513026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 37/71] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:03.514237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dfeaa-14ec-462c-ab32-521eb7994342 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.514534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dfeaa-14ec-462c-ab32-521eb7994342 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.514819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3dfeaa-14ec-462c-ab32-521eb7994342 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.516068 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 38/72] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:03.516769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3dfeaa-14ec-462c-ab32-521eb7994342 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:03.559318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 35/73] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:03.560827 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d768ebe-ee95-4800-9805-20d1a48036ba None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:03.618886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3dfeaa-14ec-462c-ab32-521eb7994342 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:03.652210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a096e98-b369-4cf4-8cd7-a62229ba88ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:03.653006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a096e98-b369-4cf4-8cd7-a62229ba88ca None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.653294 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a096e98-b369-4cf4-8cd7-a62229ba88ca None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.653571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a096e98-b369-4cf4-8cd7-a62229ba88ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.657266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d768ebe-ee95-4800-9805-20d1a48036ba None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:03.663403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a096e98-b369-4cf4-8cd7-a62229ba88ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-9a096e98-b369-4cf4-8cd7-a62229ba88ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:03.671803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:03.672843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:03.673607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:03.673607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:03.673607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:03.673607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:03.673607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:03.673607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:03.673607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:03.673607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:03.673607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:03.673607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:03.673607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:03.673607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:03.673607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 39/74] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:03.674239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a096e98-b369-4cf4-8cd7-a62229ba88ca None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:03.679216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:03.696134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101786-7f6f-4386-8431-7ab051fb50c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:03.696812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101786-7f6f-4386-8431-7ab051fb50c4 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.697103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101786-7f6f-4386-8431-7ab051fb50c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.697386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101786-7f6f-4386-8431-7ab051fb50c4 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.697796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:03.698142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d101786-7f6f-4386-8431-7ab051fb50c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:03.698446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d101786-7f6f-4386-8431-7ab051fb50c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:03.699831 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d101786-7f6f-4386-8431-7ab051fb50c4 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:03.711490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 36/75] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:03.712750 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45e044-8e56-4ae0-8ea7-f075502094f0 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.713045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45e044-8e56-4ae0-8ea7-f075502094f0 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.713328 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa45e044-8e56-4ae0-8ea7-f075502094f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.730739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-fa45e044-8e56-4ae0-8ea7-f075502094f0 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:03.731938 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-fa45e044-8e56-4ae0-8ea7-f075502094f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:03.777025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a096e98-b369-4cf4-8cd7-a62229ba88ca None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:03.799842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 37/76] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:03.800556 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45e044-8e56-4ae0-8ea7-f075502094f0 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:03.811327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035fdb6-785c-4c92-b64f-9559c46f52ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:03.812295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035fdb6-785c-4c92-b64f-9559c46f52ec None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.812587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035fdb6-785c-4c92-b64f-9559c46f52ec None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.812869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035fdb6-785c-4c92-b64f-9559c46f52ec None admin] PATH_INFO: `/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.828283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035fdb6-785c-4c92-b64f-9559c46f52ec None admin] RBAC: Authorizing `identity:get_user(user_id=7db339453aae4a3fa6c2ae5cb3a3a2eb)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:03.829749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a035fdb6-785c-4c92-b64f-9559c46f52ec None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:03.831617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 40/77] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:03 2022] GET /identity/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:03.832320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035fdb6-785c-4c92-b64f-9559c46f52ec None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:03.901810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa45e044-8e56-4ae0-8ea7-f075502094f0 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:03.911558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:03.911955 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:03.931355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035fdb6-785c-4c92-b64f-9559c46f52ec None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:03.937127 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0d5cd-53e8-4420-a9ac-a035290e87b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:03.937946 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0d5cd-53e8-4420-a9ac-a035290e87b7 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.938231 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0d5cd-53e8-4420-a9ac-a035290e87b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.938512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0d5cd-53e8-4420-a9ac-a035290e87b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.948965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd0d5cd-53e8-4420-a9ac-a035290e87b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-5bd0d5cd-53e8-4420-a9ac-a035290e87b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:03.957013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:03.957823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:03.958778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:03.958778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:03.958778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:03.958778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:03.958778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:03.958778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:03.958778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:03.958778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:03.958778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:03.958778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:03.958778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:03.958778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:03.958778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 38/78] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:37:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:03.959539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0d5cd-53e8-4420-a9ac-a035290e87b7 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:03.968993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e7849-457e-4197-bc83-491147d5a8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:03.969676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e7849-457e-4197-bc83-491147d5a8de None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:03.969971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e7849-457e-4197-bc83-491147d5a8de None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:03.970255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e7849-457e-4197-bc83-491147d5a8de None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:03.971007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e7849-457e-4197-bc83-491147d5a8de None admin] RBAC: Adding query filter params (name=member) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:03.971326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e7849-457e-4197-bc83-491147d5a8de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:03.972766 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5e7849-457e-4197-bc83-491147d5a8de None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:03.981503 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 41/79] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:03.982291 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e7849-457e-4197-bc83-491147d5a8de None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:04.058274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0d5cd-53e8-4420-a9ac-a035290e87b7 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:04.081212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5e7849-457e-4197-bc83-491147d5a8de None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:04.091404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c8262-a67d-4743-ab43-0ef0c102ec91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:04.092312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c8262-a67d-4743-ab43-0ef0c102ec91 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:04.092616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c8262-a67d-4743-ab43-0ef0c102ec91 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:04.092899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c8262-a67d-4743-ab43-0ef0c102ec91 None admin] PATH_INFO: `/v3/users/7d77dedfd6bd435d94f994cfcde5100a` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:04.094127 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c8262-a67d-4743-ab43-0ef0c102ec91 None admin] RBAC: Authorizing `identity:get_user(user_id=7d77dedfd6bd435d94f994cfcde5100a)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:04.096203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c8262-a67d-4743-ab43-0ef0c102ec91 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:04.097556 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 39/80] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:03 2022] GET /identity/v3/users/7d77dedfd6bd435d94f994cfcde5100a => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:04.098125 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c8262-a67d-4743-ab43-0ef0c102ec91 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:04.118732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781b0d4-6cf7-4204-ac5e-45641824bb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:04.119542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781b0d4-6cf7-4204-ac5e-45641824bb74 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:04.119826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781b0d4-6cf7-4204-ac5e-45641824bb74 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:04.120129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9781b0d4-6cf7-4204-ac5e-45641824bb74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:04.121292 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781b0d4-6cf7-4204-ac5e-45641824bb74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:04.122759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9781b0d4-6cf7-4204-ac5e-45641824bb74 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:04.124392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 42/81] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:37:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:04.125149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781b0d4-6cf7-4204-ac5e-45641824bb74 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:04.137960 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:04.141674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:04.199909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c8262-a67d-4743-ab43-0ef0c102ec91 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:04.227515 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9781b0d4-6cf7-4204-ac5e-45641824bb74 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:04.233142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd839829-c52d-42ac-bf61-f5a02f86e5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:04.233796 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd839829-c52d-42ac-bf61-f5a02f86e5fc None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:04.234085 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd839829-c52d-42ac-bf61-f5a02f86e5fc None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:04.234366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd839829-c52d-42ac-bf61-f5a02f86e5fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:04.261573 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a22a9f-cf2c-4a47-b0ca-3a5e89fd634c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:04.262245 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a22a9f-cf2c-4a47-b0ca-3a5e89fd634c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:04.262536 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a22a9f-cf2c-4a47-b0ca-3a5e89fd634c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:04.262815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a22a9f-cf2c-4a47-b0ca-3a5e89fd634c None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:04.263562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a22a9f-cf2c-4a47-b0ca-3a5e89fd634c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:04.263873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a22a9f-cf2c-4a47-b0ca-3a5e89fd634c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:04.265279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a22a9f-cf2c-4a47-b0ca-3a5e89fd634c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:04.273212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 43/82] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:37:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:04.274082 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a22a9f-cf2c-4a47-b0ca-3a5e89fd634c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:04.306837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd839829-c52d-42ac-bf61-f5a02f86e5fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:04.308417 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd839829-c52d-42ac-bf61-f5a02f86e5fc None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:04.344955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:04.346149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:04.375063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a22a9f-cf2c-4a47-b0ca-3a5e89fd634c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:04.407621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 40/83] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:04.408525 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd839829-c52d-42ac-bf61-f5a02f86e5fc None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:04.411338 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebe73f-8acf-41ea-9b81-59f3793391d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:04.412530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebe73f-8acf-41ea-9b81-59f3793391d3 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:04.412822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebe73f-8acf-41ea-9b81-59f3793391d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:04.413107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccebe73f-8acf-41ea-9b81-59f3793391d3 None admin] PATH_INFO: `/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:04.414331 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebe73f-8acf-41ea-9b81-59f3793391d3 None admin] RBAC: Authorizing `identity:get_user(user_id=7db339453aae4a3fa6c2ae5cb3a3a2eb)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:04.415772 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccebe73f-8acf-41ea-9b81-59f3793391d3 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:04.421828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 44/84] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:04 2022] GET /identity/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:04.421828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebe73f-8acf-41ea-9b81-59f3793391d3 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:04.507038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd839829-c52d-42ac-bf61-f5a02f86e5fc None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:04.524771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccebe73f-8acf-41ea-9b81-59f3793391d3 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:04.539830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856867c2-4c27-4f9c-8bbf-a115dfd6f9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:04.540525 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856867c2-4c27-4f9c-8bbf-a115dfd6f9a5 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:04.540819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856867c2-4c27-4f9c-8bbf-a115dfd6f9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:04.541100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856867c2-4c27-4f9c-8bbf-a115dfd6f9a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:04.550080 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:04.558700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a594d-e886-4535-a4c0-faaf706e5e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:04.559378 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a594d-e886-4535-a4c0-faaf706e5e2c None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:04.559684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a594d-e886-4535-a4c0-faaf706e5e2c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:04.559978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a594d-e886-4535-a4c0-faaf706e5e2c None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:04.560701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a594d-e886-4535-a4c0-faaf706e5e2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:04.562149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14a594d-e886-4535-a4c0-faaf706e5e2c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:04.571421 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:04.582541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 45/85] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:37:04 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:37:04.583337 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a594d-e886-4535-a4c0-faaf706e5e2c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:04.612971 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856867c2-4c27-4f9c-8bbf-a115dfd6f9a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:04.614561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856867c2-4c27-4f9c-8bbf-a115dfd6f9a5 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:04.708149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a594d-e886-4535-a4c0-faaf706e5e2c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:04.732815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 41/86] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:04.733245 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856867c2-4c27-4f9c-8bbf-a115dfd6f9a5 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:04.751028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729f94a-ebc0-4db6-876e-98401ab3c978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:04.752139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729f94a-ebc0-4db6-876e-98401ab3c978 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:04.752425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729f94a-ebc0-4db6-876e-98401ab3c978 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:04.752706 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6729f94a-ebc0-4db6-876e-98401ab3c978 None admin] PATH_INFO: `/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:04.754677 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729f94a-ebc0-4db6-876e-98401ab3c978 None admin] RBAC: Authorizing `identity:get_user(user_id=7db339453aae4a3fa6c2ae5cb3a3a2eb)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:04.756153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6729f94a-ebc0-4db6-876e-98401ab3c978 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:04.757861 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 46/87] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:04 2022] GET /identity/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:04.758574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729f94a-ebc0-4db6-876e-98401ab3c978 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:04.778751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:04.845139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:04.858646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856867c2-4c27-4f9c-8bbf-a115dfd6f9a5 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:04.899571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6729f94a-ebc0-4db6-876e-98401ab3c978 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:04.900920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c93d3e-2ba9-4603-891d-83a4f08d31ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:04.902451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c93d3e-2ba9-4603-891d-83a4f08d31ae None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:04.902753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c93d3e-2ba9-4603-891d-83a4f08d31ae None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:04.903021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c93d3e-2ba9-4603-891d-83a4f08d31ae None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:04.906013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c93d3e-2ba9-4603-891d-83a4f08d31ae None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:04.907475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c93d3e-2ba9-4603-891d-83a4f08d31ae None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:04.909741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 42/88] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:04 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:04.910041 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c93d3e-2ba9-4603-891d-83a4f08d31ae None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:04.950422 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75913f83-b222-448b-bf07-5ccf2ea5e91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:04.951875 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913f83-b222-448b-bf07-5ccf2ea5e91b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:04.952787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913f83-b222-448b-bf07-5ccf2ea5e91b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:04.953264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75913f83-b222-448b-bf07-5ccf2ea5e91b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:05.021883 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c93d3e-2ba9-4603-891d-83a4f08d31ae None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:05.030097 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:05.047679 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75913f83-b222-448b-bf07-5ccf2ea5e91b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:05.050062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75913f83-b222-448b-bf07-5ccf2ea5e91b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:05.060530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54946a78-4252-469c-b329-917c72f64d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:05.061553 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54946a78-4252-469c-b329-917c72f64d62 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:05.061948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54946a78-4252-469c-b329-917c72f64d62 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:05.062331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54946a78-4252-469c-b329-917c72f64d62 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:05.063825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54946a78-4252-469c-b329-917c72f64d62 None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:05.065401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54946a78-4252-469c-b329-917c72f64d62 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:05.067324 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 43/89] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:04 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:05.072175 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54946a78-4252-469c-b329-917c72f64d62 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:05.144340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:05.170211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 47/90] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:05.171115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75913f83-b222-448b-bf07-5ccf2ea5e91b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:05.188378 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54946a78-4252-469c-b329-917c72f64d62 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:05.225457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18841a8c-7e3c-4436-9e64-5741d1c34554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:05.226614 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18841a8c-7e3c-4436-9e64-5741d1c34554 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:05.227124 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18841a8c-7e3c-4436-9e64-5741d1c34554 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:05.227651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18841a8c-7e3c-4436-9e64-5741d1c34554 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:05.276889 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:05.297773 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75913f83-b222-448b-bf07-5ccf2ea5e91b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:05.303509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18841a8c-7e3c-4436-9e64-5741d1c34554 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:05.305279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18841a8c-7e3c-4436-9e64-5741d1c34554 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:05.340220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56438c99-7316-44dc-b1bb-7b53505e4c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:05.340896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56438c99-7316-44dc-b1bb-7b53505e4c51 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:05.341185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56438c99-7316-44dc-b1bb-7b53505e4c51 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:05.341471 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56438c99-7316-44dc-b1bb-7b53505e4c51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:05.342491 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56438c99-7316-44dc-b1bb-7b53505e4c51 None admin] RBAC: Adding query filter params (role.id=fbd0f348273f4791b1813d1df24df10e, scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82, user.id=7db339453aae4a3fa6c2ae5cb3a3a2eb) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:05.342817 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56438c99-7316-44dc-b1bb-7b53505e4c51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:05.345105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56438c99-7316-44dc-b1bb-7b53505e4c51 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:05.351573 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 48/91] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:05 2022] GET /identity/v3/role_assignments?user.id=7db339453aae4a3fa6c2ae5cb3a3a2eb&scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82&role.id=fbd0f348273f4791b1813d1df24df10e => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:05.353020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56438c99-7316-44dc-b1bb-7b53505e4c51 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:05.391216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:05.440964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 44/92] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:05.442261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18841a8c-7e3c-4436-9e64-5741d1c34554 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:05.485393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56438c99-7316-44dc-b1bb-7b53505e4c51 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:05.531533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d4d59-06cb-46d5-be54-2e2ed716a24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:05.532171 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d4d59-06cb-46d5-be54-2e2ed716a24c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:05.532467 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d4d59-06cb-46d5-be54-2e2ed716a24c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:05.532757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481d4d59-06cb-46d5-be54-2e2ed716a24c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:05.533616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d4d59-06cb-46d5-be54-2e2ed716a24c None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82, user.id=7d77dedfd6bd435d94f994cfcde5100a) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:05.533948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d4d59-06cb-46d5-be54-2e2ed716a24c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:05.537856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481d4d59-06cb-46d5-be54-2e2ed716a24c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:05.543647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 49/93] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:05 2022] GET /identity/v3/role_assignments?user.id=7d77dedfd6bd435d94f994cfcde5100a&scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:05.544399 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d4d59-06cb-46d5-be54-2e2ed716a24c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:05.623920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:05.680847 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18841a8c-7e3c-4436-9e64-5741d1c34554 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:05.747644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407d7b3-8b01-485b-bcea-43bee6bea400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:05.749743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407d7b3-8b01-485b-bcea-43bee6bea400 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:05.752860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407d7b3-8b01-485b-bcea-43bee6bea400 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:05.753378 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407d7b3-8b01-485b-bcea-43bee6bea400 None admin] PATH_INFO: `/v3/projects/8994e6acc2c242949ceaf4437c49317e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:05.754891 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407d7b3-8b01-485b-bcea-43bee6bea400 None admin] RBAC: Authorizing `identity:get_project(project_id=8994e6acc2c242949ceaf4437c49317e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:05.756731 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407d7b3-8b01-485b-bcea-43bee6bea400 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:05.761572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 45/94] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:05 2022] GET /identity/v3/projects/8994e6acc2c242949ceaf4437c49317e => generated 312 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:05.762711 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:05.766951 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407d7b3-8b01-485b-bcea-43bee6bea400 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:05.781894 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481d4d59-06cb-46d5-be54-2e2ed716a24c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:05.826050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84ce46-46b9-476b-bb09-52d39b6b5de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:05.827383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84ce46-46b9-476b-bb09-52d39b6b5de8 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:05.827875 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84ce46-46b9-476b-bb09-52d39b6b5de8 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:05.828413 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84ce46-46b9-476b-bb09-52d39b6b5de8 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:05.829841 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84ce46-46b9-476b-bb09-52d39b6b5de8 None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:05.831507 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84ce46-46b9-476b-bb09-52d39b6b5de8 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:05.833683 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 50/95] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:05 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:05.841006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84ce46-46b9-476b-bb09-52d39b6b5de8 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:05.960028 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407d7b3-8b01-485b-bcea-43bee6bea400 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:06.024224 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99d34e-955e-4ce3-8585-4beadb26caed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:06.024922 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99d34e-955e-4ce3-8585-4beadb26caed None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:06.025188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99d34e-955e-4ce3-8585-4beadb26caed None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:06.025463 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99d34e-955e-4ce3-8585-4beadb26caed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:06.026364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99d34e-955e-4ce3-8585-4beadb26caed None admin] RBAC: Adding query filter params (role.id=fbd0f348273f4791b1813d1df24df10e, scope.project.id=8994e6acc2c242949ceaf4437c49317e, user.id=7db339453aae4a3fa6c2ae5cb3a3a2eb) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:06.026680 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99d34e-955e-4ce3-8585-4beadb26caed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:06.028154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99d34e-955e-4ce3-8585-4beadb26caed None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:06.042473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 46/96] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:05 2022] GET /identity/v3/role_assignments?user.id=7db339453aae4a3fa6c2ae5cb3a3a2eb&scope.project.id=8994e6acc2c242949ceaf4437c49317e&role.id=fbd0f348273f4791b1813d1df24df10e => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:06.052285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84ce46-46b9-476b-bb09-52d39b6b5de8 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:06.124959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4697ffd-37b8-4dfa-9bb8-c1dab1b85484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:06.125704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4697ffd-37b8-4dfa-9bb8-c1dab1b85484 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:06.125998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4697ffd-37b8-4dfa-9bb8-c1dab1b85484 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:06.126287 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4697ffd-37b8-4dfa-9bb8-c1dab1b85484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:06.127153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4697ffd-37b8-4dfa-9bb8-c1dab1b85484 None admin] RBAC: Adding query filter params (role.id=8427479f632a4da3949971f58b438c7f, scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82, user.id=7db339453aae4a3fa6c2ae5cb3a3a2eb) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:06.127477 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4697ffd-37b8-4dfa-9bb8-c1dab1b85484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:06.129052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4697ffd-37b8-4dfa-9bb8-c1dab1b85484 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:06.141410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 51/97] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:05 2022] GET /identity/v3/role_assignments?user.id=7db339453aae4a3fa6c2ae5cb3a3a2eb&scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82&role.id=8427479f632a4da3949971f58b438c7f => generated 259 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:06.856527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19784b6-2544-409d-a138-57a97d88d0c1 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:06.856900 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19784b6-2544-409d-a138-57a97d88d0c1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:06.857091 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19784b6-2544-409d-a138-57a97d88d0c1 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:06.858471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 47/98] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:06.863880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4705b-dcde-4daf-91de-8572d0819a08 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:06.864210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4705b-dcde-4daf-91de-8572d0819a08 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:06.864501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4705b-dcde-4daf-91de-8572d0819a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:06.887045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-bda4705b-dcde-4daf-91de-8572d0819a08 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:06.888272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-bda4705b-dcde-4daf-91de-8572d0819a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:06.916314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:06.987795 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 52/99] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:07.020096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19784b6-2544-409d-a138-57a97d88d0c1 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:07.029615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:07.153554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19784b6-2544-409d-a138-57a97d88d0c1 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:07.195750 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7b59b-0ce5-4651-8ec0-dfa2c2f150a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:07.196590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7b59b-0ce5-4651-8ec0-dfa2c2f150a8 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:07.196879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7b59b-0ce5-4651-8ec0-dfa2c2f150a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:07.197160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7b59b-0ce5-4651-8ec0-dfa2c2f150a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:07.213029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7b59b-0ce5-4651-8ec0-dfa2c2f150a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-05e7b59b-0ce5-4651-8ec0-dfa2c2f150a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:07.221585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:07.222434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:07.223205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:07.223205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:07.223205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:07.223205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:07.223205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:07.223205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:07.223205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:07.223205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:07.223205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:07.223205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:07.223205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:07.223205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:07.223205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 48/100] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:07.230516 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4705b-dcde-4daf-91de-8572d0819a08 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:07.371727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4705b-dcde-4daf-91de-8572d0819a08 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:07.420297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffc968-f8c1-4243-8beb-f93da35d9194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:07.421390 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffc968-f8c1-4243-8beb-f93da35d9194 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:07.421880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffc968-f8c1-4243-8beb-f93da35d9194 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:07.422355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffc968-f8c1-4243-8beb-f93da35d9194 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:07.423305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffc968-f8c1-4243-8beb-f93da35d9194 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:07.423814 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffc968-f8c1-4243-8beb-f93da35d9194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:07.425432 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffc968-f8c1-4243-8beb-f93da35d9194 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:07.436937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 53/101] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:07.442696 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7b59b-0ce5-4651-8ec0-dfa2c2f150a8 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:07.515447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:07.561533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7b59b-0ce5-4651-8ec0-dfa2c2f150a8 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:07.604118 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0ab3d-5fef-4459-a6cd-e0d55bb18780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:07.605608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0ab3d-5fef-4459-a6cd-e0d55bb18780 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:07.605893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0ab3d-5fef-4459-a6cd-e0d55bb18780 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:07.606171 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0ab3d-5fef-4459-a6cd-e0d55bb18780 None admin] PATH_INFO: `/v3/users/c3d7911f2f5f4f5386cd51a788182aa7` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:07.632098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423fb69-a35d-454e-a859-aa92299a442f None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:07.632669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0ab3d-5fef-4459-a6cd-e0d55bb18780 None admin] RBAC: Authorizing `identity:get_user(user_id=c3d7911f2f5f4f5386cd51a788182aa7)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:07.634169 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0ab3d-5fef-4459-a6cd-e0d55bb18780 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:07.634970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423fb69-a35d-454e-a859-aa92299a442f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:07.635463 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423fb69-a35d-454e-a859-aa92299a442f None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:07.635975 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 49/102] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:07 2022] GET /identity/v3/users/c3d7911f2f5f4f5386cd51a788182aa7 => generated 289 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:07.637369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 54/103] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:37:07.642748 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b964e-d5a9-4141-bab4-a673d4473e7d None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:07.642976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423fb69-a35d-454e-a859-aa92299a442f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:07.643075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b964e-d5a9-4141-bab4-a673d4473e7d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:07.643314 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367b964e-d5a9-4141-bab4-a673d4473e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:07.665252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-367b964e-d5a9-4141-bab4-a673d4473e7d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:07.666429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-367b964e-d5a9-4141-bab4-a673d4473e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:07.676038 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:07.777958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 50/104] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:07.844618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423fb69-a35d-454e-a859-aa92299a442f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:07.865012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b964e-d5a9-4141-bab4-a673d4473e7d None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:07.895043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54e17b-69b7-476c-85b1-f2916e2588f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:07.895618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54e17b-69b7-476c-85b1-f2916e2588f0 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:07.895911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54e17b-69b7-476c-85b1-f2916e2588f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:07.896227 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab54e17b-69b7-476c-85b1-f2916e2588f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:07.951992 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:07.998322 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54e17b-69b7-476c-85b1-f2916e2588f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:07.999828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab54e17b-69b7-476c-85b1-f2916e2588f0 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:08.017078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:08.037507 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367b964e-d5a9-4141-bab4-a673d4473e7d None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:08.117858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d2df5-59c9-4db4-9ac7-c0683c1d46c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:08.117858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d2df5-59c9-4db4-9ac7-c0683c1d46c2 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:08.117858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d2df5-59c9-4db4-9ac7-c0683c1d46c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:08.118198 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8d2df5-59c9-4db4-9ac7-c0683c1d46c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:08.123986 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d2df5-59c9-4db4-9ac7-c0683c1d46c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:08.126876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8d2df5-59c9-4db4-9ac7-c0683c1d46c2 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:08.126876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 51/105] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:37:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:08.142365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 55/106] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 17:37:08.144998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d2df5-59c9-4db4-9ac7-c0683c1d46c2 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:08.148914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54e17b-69b7-476c-85b1-f2916e2588f0 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:08.323115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab54e17b-69b7-476c-85b1-f2916e2588f0 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:08.348383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:08.354144 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8d2df5-59c9-4db4-9ac7-c0683c1d46c2 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:08.368870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf3d30-718f-4e67-8f86-80481482ebf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:08.369835 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf3d30-718f-4e67-8f86-80481482ebf4 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:08.370153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf3d30-718f-4e67-8f86-80481482ebf4 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:08.370437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf3d30-718f-4e67-8f86-80481482ebf4 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:08.371634 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf3d30-718f-4e67-8f86-80481482ebf4 None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:08.373304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf3d30-718f-4e67-8f86-80481482ebf4 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:08.376509 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 56/107] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:08 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:08.381455 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d06a1-b1c3-4844-8288-62a7c276b9d2 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:08.381954 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d06a1-b1c3-4844-8288-62a7c276b9d2 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:08.382452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0d06a1-b1c3-4844-8288-62a7c276b9d2 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:08.384058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 57/108] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:08.385006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d06a1-b1c3-4844-8288-62a7c276b9d2 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:08.408442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:08.417592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c302878-88d6-40f4-9622-d04faccf826d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:08.418279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c302878-88d6-40f4-9622-d04faccf826d None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:08.418678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c302878-88d6-40f4-9622-d04faccf826d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:08.418841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c302878-88d6-40f4-9622-d04faccf826d None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:08.419573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c302878-88d6-40f4-9622-d04faccf826d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:08.421092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c302878-88d6-40f4-9622-d04faccf826d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:08.451036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 52/109] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Jul 14 17:37:08 2022] POST /identity/v3/projects => generated 302 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:37:08.452940 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce79eee-5426-4182-a137-7a6832b9e537 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:08.453253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce79eee-5426-4182-a137-7a6832b9e537 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:08.453548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce79eee-5426-4182-a137-7a6832b9e537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:08.474258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-dce79eee-5426-4182-a137-7a6832b9e537 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:08.475490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-dce79eee-5426-4182-a137-7a6832b9e537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:08.556521 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 53/110] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:08.582296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce79eee-5426-4182-a137-7a6832b9e537 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:08.647357 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0d06a1-b1c3-4844-8288-62a7c276b9d2 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:08.705372 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d784235-b6a9-4ed2-ab33-8dc7677a0562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:08.706893 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d784235-b6a9-4ed2-ab33-8dc7677a0562 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:08.706893 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d784235-b6a9-4ed2-ab33-8dc7677a0562 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:08.706893 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d784235-b6a9-4ed2-ab33-8dc7677a0562 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:08.707472 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d784235-b6a9-4ed2-ab33-8dc7677a0562 None admin] RBAC: Adding query filter params (role.id=84bd13e4cee04c12bd6a0e549db79246, scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82, user.id=c3d7911f2f5f4f5386cd51a788182aa7) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:08.707796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d784235-b6a9-4ed2-ab33-8dc7677a0562 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:08.709977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d784235-b6a9-4ed2-ab33-8dc7677a0562 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:08.716823 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 58/111] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:08 2022] GET /identity/v3/role_assignments?user.id=c3d7911f2f5f4f5386cd51a788182aa7&scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82&role.id=84bd13e4cee04c12bd6a0e549db79246 => generated 259 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:08.724272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d32cc-45a6-434d-9cd1-6728f4e3719e None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:08.724692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d32cc-45a6-434d-9cd1-6728f4e3719e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:08.724986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d32cc-45a6-434d-9cd1-6728f4e3719e None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:08.726470 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 59/112] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:08 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:08.736877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402823d-193e-4fbf-aeef-fa12afa83c99 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:08.737457 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402823d-193e-4fbf-aeef-fa12afa83c99 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:08.737924 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402823d-193e-4fbf-aeef-fa12afa83c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:08.773806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b402823d-193e-4fbf-aeef-fa12afa83c99 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:08.775192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b402823d-193e-4fbf-aeef-fa12afa83c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:08.777239 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce79eee-5426-4182-a137-7a6832b9e537 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:08.780245 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:08.824099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97881a4d-ea32-405d-87c5-232270652528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:08.824923 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97881a4d-ea32-405d-87c5-232270652528 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:08.825214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97881a4d-ea32-405d-87c5-232270652528 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:08.825497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97881a4d-ea32-405d-87c5-232270652528 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:08.849052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97881a4d-ea32-405d-87c5-232270652528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-97881a4d-ea32-405d-87c5-232270652528 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:08.857596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:08.858510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:08.859308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:08.859308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:08.859308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:08.859308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:08.859308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:08.859308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:08.859308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:08.859308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:08.859308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:08.859308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:08.859308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:08.859308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:08.859308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 54/113] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:37:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:08.869023 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97881a4d-ea32-405d-87c5-232270652528 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:08.872230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:08.920700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 60/114] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:09.007200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402823d-193e-4fbf-aeef-fa12afa83c99 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:09.015386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97881a4d-ea32-405d-87c5-232270652528 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:09.071748 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e8464-e3c4-4b89-8a3e-a91a73f37c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:09.073276 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e8464-e3c4-4b89-8a3e-a91a73f37c26 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:09.074190 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e8464-e3c4-4b89-8a3e-a91a73f37c26 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:09.074591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e8464-e3c4-4b89-8a3e-a91a73f37c26 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:09.075469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e8464-e3c4-4b89-8a3e-a91a73f37c26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:09.075898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e8464-e3c4-4b89-8a3e-a91a73f37c26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:09.077452 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e8464-e3c4-4b89-8a3e-a91a73f37c26 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:09.091600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 55/115] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:09.097328 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e8464-e3c4-4b89-8a3e-a91a73f37c26 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:09.149663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:09.196704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402823d-193e-4fbf-aeef-fa12afa83c99 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:09.270089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16a51d-f51d-4664-a299-17ee7f411f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:09.270836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16a51d-f51d-4664-a299-17ee7f411f7d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:09.271130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16a51d-f51d-4664-a299-17ee7f411f7d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:09.271416 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a16a51d-f51d-4664-a299-17ee7f411f7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:09.275291 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e8464-e3c4-4b89-8a3e-a91a73f37c26 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:09.297627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a16a51d-f51d-4664-a299-17ee7f411f7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-3a16a51d-f51d-4664-a299-17ee7f411f7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:09.317787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:09.318686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:09.319458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:09.319458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:09.319458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:09.319458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:09.319458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:09.319458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:09.319458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:09.319458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:09.319458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:09.319458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:09.319458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:09.319458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:09.319458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 61/116] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:09.328918 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16a51d-f51d-4664-a299-17ee7f411f7d None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:09.341607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133d734-2c17-4aa8-95f1-afa8827453b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:09.342503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133d734-2c17-4aa8-95f1-afa8827453b8 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:09.342786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133d734-2c17-4aa8-95f1-afa8827453b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:09.343061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133d734-2c17-4aa8-95f1-afa8827453b8 None admin] PATH_INFO: `/v3/users/7d77dedfd6bd435d94f994cfcde5100a` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:09.348766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133d734-2c17-4aa8-95f1-afa8827453b8 None admin] RBAC: Authorizing `identity:get_user(user_id=7d77dedfd6bd435d94f994cfcde5100a)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:09.350266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133d734-2c17-4aa8-95f1-afa8827453b8 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:09.351985 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 56/117] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:09 2022] GET /identity/v3/users/7d77dedfd6bd435d94f994cfcde5100a => generated 247 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:09.358280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133d734-2c17-4aa8-95f1-afa8827453b8 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:09.396687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:09.472114 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a16a51d-f51d-4664-a299-17ee7f411f7d None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:09.485685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:09.497599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133d734-2c17-4aa8-95f1-afa8827453b8 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:09.519428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51618f56-c407-4e06-bce6-3935255b6ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:09.520200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51618f56-c407-4e06-bce6-3935255b6ef8 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:09.520422 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51618f56-c407-4e06-bce6-3935255b6ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:09.520713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51618f56-c407-4e06-bce6-3935255b6ef8 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:09.521760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51618f56-c407-4e06-bce6-3935255b6ef8 None admin] RBAC: Adding query filter params (name=member) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:09.522074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51618f56-c407-4e06-bce6-3935255b6ef8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:09.523463 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51618f56-c407-4e06-bce6-3935255b6ef8 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:09.534391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 62/118] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:09.538986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b569dde-4302-4d8b-b65c-a4e3ce0ddb8a None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:09.538986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b569dde-4302-4d8b-b65c-a4e3ce0ddb8a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:09.538986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b569dde-4302-4d8b-b65c-a4e3ce0ddb8a None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:09.538986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 63/119] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:09.538986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecb001-ffa2-43e4-8735-1304a6b62c65 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:09.538986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecb001-ffa2-43e4-8735-1304a6b62c65 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:09.543837 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecb001-ffa2-43e4-8735-1304a6b62c65 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:09.545497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 64/120] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:09.548022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecb001-ffa2-43e4-8735-1304a6b62c65 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:09.555538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d5dcf-4f4d-4872-969c-3af0aaa8459c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:09.557158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d5dcf-4f4d-4872-969c-3af0aaa8459c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:09.558181 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d5dcf-4f4d-4872-969c-3af0aaa8459c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:09.559015 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d5dcf-4f4d-4872-969c-3af0aaa8459c None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:09.560798 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d5dcf-4f4d-4872-969c-3af0aaa8459c None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:09.562735 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d5dcf-4f4d-4872-969c-3af0aaa8459c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:09.570317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 57/121] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:09 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:09.572120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b374a7e-4234-4f74-a273-d55aabe246bf None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:09.572613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b374a7e-4234-4f74-a273-d55aabe246bf None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:09.573102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b374a7e-4234-4f74-a273-d55aabe246bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:09.593989 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-2b374a7e-4234-4f74-a273-d55aabe246bf None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:09.595861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-2b374a7e-4234-4f74-a273-d55aabe246bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:09.662560 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:09.667557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecb001-ffa2-43e4-8735-1304a6b62c65 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:09.690137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 58/122] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:09.692838 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a14a-576e-4aaa-a88d-0a15a7f65424 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:09.693351 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a14a-576e-4aaa-a88d-0a15a7f65424 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:09.693937 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3a14a-576e-4aaa-a88d-0a15a7f65424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:09.715494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-c2b3a14a-576e-4aaa-a88d-0a15a7f65424 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:09.717478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c2b3a14a-576e-4aaa-a88d-0a15a7f65424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:09.724573 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb89cc-d4e7-4c8a-9875-5a46a2f1070b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:09.725667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb89cc-d4e7-4c8a-9875-5a46a2f1070b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:09.726046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb89cc-d4e7-4c8a-9875-5a46a2f1070b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:09.726400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb89cc-d4e7-4c8a-9875-5a46a2f1070b None admin] PATH_INFO: `/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:09.727792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb89cc-d4e7-4c8a-9875-5a46a2f1070b None admin] RBAC: Authorizing `identity:get_user(user_id=7db339453aae4a3fa6c2ae5cb3a3a2eb)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:09.730019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb89cc-d4e7-4c8a-9875-5a46a2f1070b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:09.734948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 65/123] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:09 2022] GET /identity/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:09.735655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb89cc-d4e7-4c8a-9875-5a46a2f1070b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:09.808593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 59/124] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:09.811974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3a14a-576e-4aaa-a88d-0a15a7f65424 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:09.846393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:09.881917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb89cc-d4e7-4c8a-9875-5a46a2f1070b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:09.925309 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d170ce-8845-45fd-8371-b44d8c9154f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:09.926862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d170ce-8845-45fd-8371-b44d8c9154f8 None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:09.927229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d170ce-8845-45fd-8371-b44d8c9154f8 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:09.927586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d170ce-8845-45fd-8371-b44d8c9154f8 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82/users/7d77dedfd6bd435d94f994cfcde5100a/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:09.928646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3a14a-576e-4aaa-a88d-0a15a7f65424 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:09.929782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d170ce-8845-45fd-8371-b44d8c9154f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba02b19bd80d44c4a9a9ae6517b11b82, user_id=7d77dedfd6bd435d94f994cfcde5100a, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:09.932250 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d170ce-8845-45fd-8371-b44d8c9154f8 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:09.950325 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 66/125] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:37:09 2022] PUT /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82/users/7d77dedfd6bd435d94f994cfcde5100a/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:37:09.958202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d170ce-8845-45fd-8371-b44d8c9154f8 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:09.987246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a13181-2695-4a71-b51e-406ac51563eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:09.988147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a13181-2695-4a71-b51e-406ac51563eb None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:09.988432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a13181-2695-4a71-b51e-406ac51563eb None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:09.988708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a13181-2695-4a71-b51e-406ac51563eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:10.016464 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a13181-2695-4a71-b51e-406ac51563eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:10.032281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-55a13181-2695-4a71-b51e-406ac51563eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:10.033337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:10.034197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:10.035049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:10.035049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:10.035049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:10.035049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:10.035049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:10.035049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:10.035049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:10.035049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:10.035049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:10.035049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:10.035049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:10.035049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:10.036750 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 60/126] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:10.037410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a13181-2695-4a71-b51e-406ac51563eb None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:10.175137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:10.191380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d170ce-8845-45fd-8371-b44d8c9154f8 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:10.205948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a13181-2695-4a71-b51e-406ac51563eb None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:10.252079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1deb8-0e99-4270-a960-e98711692d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:10.252909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1deb8-0e99-4270-a960-e98711692d0a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:10.253192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1deb8-0e99-4270-a960-e98711692d0a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:10.253467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c1deb8-0e99-4270-a960-e98711692d0a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:10.265314 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c1deb8-0e99-4270-a960-e98711692d0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-f4c1deb8-0e99-4270-a960-e98711692d0a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:10.273642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:10.274497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:10.275265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:10.275265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:10.275265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:10.275265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:10.275265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:10.275265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:10.275265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:10.275265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:10.275265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:10.275265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:10.275265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:10.275265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:10.275265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 61/127] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:37:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:10.275893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1deb8-0e99-4270-a960-e98711692d0a None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:10.280738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ed650-ebec-4881-a2fb-6bec24c4c05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:10.281651 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ed650-ebec-4881-a2fb-6bec24c4c05f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:10.281946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ed650-ebec-4881-a2fb-6bec24c4c05f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:10.282233 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ed650-ebec-4881-a2fb-6bec24c4c05f None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:10.283503 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ed650-ebec-4881-a2fb-6bec24c4c05f None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:10.285014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ed650-ebec-4881-a2fb-6bec24c4c05f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:10.286744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 67/128] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:09 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:10.290746 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ed650-ebec-4881-a2fb-6bec24c4c05f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:10.386737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c1deb8-0e99-4270-a960-e98711692d0a None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:10.422266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9bc8c-d1f1-4ebe-a283-0ae5bd070f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:10.422993 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9bc8c-d1f1-4ebe-a283-0ae5bd070f65 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:10.423286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9bc8c-d1f1-4ebe-a283-0ae5bd070f65 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:10.423568 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c9bc8c-d1f1-4ebe-a283-0ae5bd070f65 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:10.424358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9bc8c-d1f1-4ebe-a283-0ae5bd070f65 None admin] RBAC: Adding query filter params (name=member) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:10.425794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9bc8c-d1f1-4ebe-a283-0ae5bd070f65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:10.426055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c9bc8c-d1f1-4ebe-a283-0ae5bd070f65 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:10.434553 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 62/129] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:10.435318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9bc8c-d1f1-4ebe-a283-0ae5bd070f65 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:10.438629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:10.524545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:10.540370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ed650-ebec-4881-a2fb-6bec24c4c05f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:10.543145 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c9bc8c-d1f1-4ebe-a283-0ae5bd070f65 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:10.576664 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae7efd-27f0-485f-aa1e-c6422529e16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:10.577342 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae7efd-27f0-485f-aa1e-c6422529e16d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:10.577633 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae7efd-27f0-485f-aa1e-c6422529e16d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:10.577918 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ae7efd-27f0-485f-aa1e-c6422529e16d None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:10.578681 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae7efd-27f0-485f-aa1e-c6422529e16d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:10.579016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae7efd-27f0-485f-aa1e-c6422529e16d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:10.580637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ae7efd-27f0-485f-aa1e-c6422529e16d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:10.581429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b1558-0e71-4a93-b5e1-c8ccf296121f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:10.582540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b1558-0e71-4a93-b5e1-c8ccf296121f None admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:10.583017 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b1558-0e71-4a93-b5e1-c8ccf296121f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:10.583503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b1558-0e71-4a93-b5e1-c8ccf296121f None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82/users/7db339453aae4a3fa6c2ae5cb3a3a2eb/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:10.585725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b1558-0e71-4a93-b5e1-c8ccf296121f None admin] RBAC: Authorizing `identity:create_grant(project_id=ba02b19bd80d44c4a9a9ae6517b11b82, user_id=7db339453aae4a3fa6c2ae5cb3a3a2eb, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:10.587344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b1558-0e71-4a93-b5e1-c8ccf296121f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:10.592154 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 68/130] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:37:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:10.592853 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae7efd-27f0-485f-aa1e-c6422529e16d None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:10.620040 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 63/131] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:37:10 2022] PUT /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82/users/7db339453aae4a3fa6c2ae5cb3a3a2eb/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:37:10.620040 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b1558-0e71-4a93-b5e1-c8ccf296121f None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:10.695771 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:10.747020 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b1558-0e71-4a93-b5e1-c8ccf296121f None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:10.790719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4a93f-8818-491f-a73c-b4f74c29a66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:10.792456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4a93f-8818-491f-a73c-b4f74c29a66a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:10.792895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4a93f-8818-491f-a73c-b4f74c29a66a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:10.793322 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4a93f-8818-491f-a73c-b4f74c29a66a None admin] PATH_INFO: `/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:10.795597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4a93f-8818-491f-a73c-b4f74c29a66a None admin] RBAC: Authorizing `identity:get_user(user_id=7db339453aae4a3fa6c2ae5cb3a3a2eb)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:10.797289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4a93f-8818-491f-a73c-b4f74c29a66a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:10.799214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 64/132] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:10 2022] GET /identity/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:10.805191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4a93f-8818-491f-a73c-b4f74c29a66a None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:10.871371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ae7efd-27f0-485f-aa1e-c6422529e16d None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:10.914150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021d570-32c0-46f4-abd5-8e0cb763475b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:10.915039 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021d570-32c0-46f4-abd5-8e0cb763475b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:10.915321 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021d570-32c0-46f4-abd5-8e0cb763475b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:10.915596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5021d570-32c0-46f4-abd5-8e0cb763475b None admin] PATH_INFO: `/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:10.917377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021d570-32c0-46f4-abd5-8e0cb763475b None admin] RBAC: Authorizing `identity:get_user(user_id=7db339453aae4a3fa6c2ae5cb3a3a2eb)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:10.918833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5021d570-32c0-46f4-abd5-8e0cb763475b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:10.920515 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 69/133] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:10 2022] GET /identity/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb => generated 275 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:10.928774 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021d570-32c0-46f4-abd5-8e0cb763475b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:10.947053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4a93f-8818-491f-a73c-b4f74c29a66a None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:10.987339 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:11.011401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:11.020364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a0858-ae15-49e4-bd3e-d587a167887c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:11.021460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a0858-ae15-49e4-bd3e-d587a167887c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:11.021896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a0858-ae15-49e4-bd3e-d587a167887c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:11.022322 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50a0858-ae15-49e4-bd3e-d587a167887c None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:11.024419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a0858-ae15-49e4-bd3e-d587a167887c None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:11.028428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50a0858-ae15-49e4-bd3e-d587a167887c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:11.032388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 65/134] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:10 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:11.048094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a0858-ae15-49e4-bd3e-d587a167887c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:11.133729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5021d570-32c0-46f4-abd5-8e0cb763475b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:11.203371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566404c-3c83-432a-992a-90ea509a45e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:11.205395 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566404c-3c83-432a-992a-90ea509a45e6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:11.206646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566404c-3c83-432a-992a-90ea509a45e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:11.207291 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566404c-3c83-432a-992a-90ea509a45e6 None admin] PATH_INFO: `/v3/projects/8994e6acc2c242949ceaf4437c49317e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:11.209874 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566404c-3c83-432a-992a-90ea509a45e6 None admin] RBAC: Authorizing `identity:get_project(project_id=8994e6acc2c242949ceaf4437c49317e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:11.214682 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566404c-3c83-432a-992a-90ea509a45e6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:11.218093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 70/135] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:10 2022] GET /identity/v3/projects/8994e6acc2c242949ceaf4437c49317e => generated 312 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:11.224450 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566404c-3c83-432a-992a-90ea509a45e6 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:11.248086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50a0858-ae15-49e4-bd3e-d587a167887c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:11.316394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc551615-c775-4134-b24f-6445ba40e52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:11.317689 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc551615-c775-4134-b24f-6445ba40e52c None admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:11.318205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc551615-c775-4134-b24f-6445ba40e52c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:11.318836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc551615-c775-4134-b24f-6445ba40e52c None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82/users/7db339453aae4a3fa6c2ae5cb3a3a2eb/roles/8427479f632a4da3949971f58b438c7f` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:11.324660 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc551615-c775-4134-b24f-6445ba40e52c None admin] RBAC: Authorizing `identity:create_grant(project_id=ba02b19bd80d44c4a9a9ae6517b11b82, user_id=7db339453aae4a3fa6c2ae5cb3a3a2eb, role_id=8427479f632a4da3949971f58b438c7f)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:11.327274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc551615-c775-4134-b24f-6445ba40e52c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:11.361452 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 66/136] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:37:11 2022] PUT /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82/users/7db339453aae4a3fa6c2ae5cb3a3a2eb/roles/8427479f632a4da3949971f58b438c7f => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:37:11.392816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:11.402119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea721d-5b19-4e78-a6e2-33a9c1618aa2 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:11.402777 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea721d-5b19-4e78-a6e2-33a9c1618aa2 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:11.403262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea721d-5b19-4e78-a6e2-33a9c1618aa2 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:11.404897 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 67/137] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:37:11.405351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566404c-3c83-432a-992a-90ea509a45e6 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:11.413517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977d9a6-cd2a-49de-b69c-4f9a0e85f6f1 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:11.414122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977d9a6-cd2a-49de-b69c-4f9a0e85f6f1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:11.414604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d977d9a6-cd2a-49de-b69c-4f9a0e85f6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:11.437665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-d977d9a6-cd2a-49de-b69c-4f9a0e85f6f1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:11.439266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d977d9a6-cd2a-49de-b69c-4f9a0e85f6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:11.482050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de3abee-31ae-495d-8b35-9e6931261d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:11.484272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3abee-31ae-495d-8b35-9e6931261d08 None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:11.485454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3abee-31ae-495d-8b35-9e6931261d08 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:11.486492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de3abee-31ae-495d-8b35-9e6931261d08 None admin] PATH_INFO: `/v3/projects/8994e6acc2c242949ceaf4437c49317e/users/7db339453aae4a3fa6c2ae5cb3a3a2eb/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:11.492147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3abee-31ae-495d-8b35-9e6931261d08 None admin] RBAC: Authorizing `identity:create_grant(project_id=8994e6acc2c242949ceaf4437c49317e, user_id=7db339453aae4a3fa6c2ae5cb3a3a2eb, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:11.495346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de3abee-31ae-495d-8b35-9e6931261d08 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:11.522851 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 71/138] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:37:11 2022] PUT /identity/v3/projects/8994e6acc2c242949ceaf4437c49317e/users/7db339453aae4a3fa6c2ae5cb3a3a2eb/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:37:11.524780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3c81e-1bdf-4bf9-9f69-9b162bbcee80 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:11.531957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3c81e-1bdf-4bf9-9f69-9b162bbcee80 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:11.533671 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3c81e-1bdf-4bf9-9f69-9b162bbcee80 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:11.537270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 72/139] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:11 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:11.543064 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee72717-ff95-4bc4-b0a4-c6171b9ad557 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:11.543616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee72717-ff95-4bc4-b0a4-c6171b9ad557 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:11.544149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee72717-ff95-4bc4-b0a4-c6171b9ad557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:11.556342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:11.578475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-dee72717-ff95-4bc4-b0a4-c6171b9ad557 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:11.579717 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-dee72717-ff95-4bc4-b0a4-c6171b9ad557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:11.584393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 68/140] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 14 17:37:11.612289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977d9a6-cd2a-49de-b69c-4f9a0e85f6f1 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:11.738299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 73/141] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:11.766949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d977d9a6-cd2a-49de-b69c-4f9a0e85f6f1 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:11.781389 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee72717-ff95-4bc4-b0a4-c6171b9ad557 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:11.819815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e5db3-4da0-47b3-812e-0fbd5a38e7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:11.821138 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e5db3-4da0-47b3-812e-0fbd5a38e7cc None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:11.821429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e5db3-4da0-47b3-812e-0fbd5a38e7cc None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:11.821857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e5db3-4da0-47b3-812e-0fbd5a38e7cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:11.839893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e5db3-4da0-47b3-812e-0fbd5a38e7cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-730e5db3-4da0-47b3-812e-0fbd5a38e7cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:11.856538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:11.857424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:11.858172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:11.858172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:11.858172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:11.858172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:11.858172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:11.858172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:11.858172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:11.858172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:11.858172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:11.858172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:11.858172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:11.858172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:11.858172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 69/142] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:11.865049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e5db3-4da0-47b3-812e-0fbd5a38e7cc None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:11.923574 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:11.938837 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:11.972806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee72717-ff95-4bc4-b0a4-c6171b9ad557 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:12.008823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e5db3-4da0-47b3-812e-0fbd5a38e7cc None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:12.072308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e999b9-c86f-4410-a4c7-347094582a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:12.073868 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e999b9-c86f-4410-a4c7-347094582a51 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:12.074271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e999b9-c86f-4410-a4c7-347094582a51 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:12.074558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e999b9-c86f-4410-a4c7-347094582a51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:12.079899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17f5a0-aaba-41fb-bd6e-a7b55f1bd6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:12.080604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17f5a0-aaba-41fb-bd6e-a7b55f1bd6f4 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:12.080702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e999b9-c86f-4410-a4c7-347094582a51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:12.080893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17f5a0-aaba-41fb-bd6e-a7b55f1bd6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:12.081172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17f5a0-aaba-41fb-bd6e-a7b55f1bd6f4 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:12.081939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17f5a0-aaba-41fb-bd6e-a7b55f1bd6f4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:12.082151 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e999b9-c86f-4410-a4c7-347094582a51 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:12.082242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17f5a0-aaba-41fb-bd6e-a7b55f1bd6f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:12.083667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17f5a0-aaba-41fb-bd6e-a7b55f1bd6f4 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:12.083844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 74/143] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:37:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:12.089296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e999b9-c86f-4410-a4c7-347094582a51 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:12.099114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 70/144] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:12.105395 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17f5a0-aaba-41fb-bd6e-a7b55f1bd6f4 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:12.229126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17f5a0-aaba-41fb-bd6e-a7b55f1bd6f4 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:12.248306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:12.295236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38525f-9345-4cdc-b928-e7ef3a910bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:12.296496 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38525f-9345-4cdc-b928-e7ef3a910bc5 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:12.297042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38525f-9345-4cdc-b928-e7ef3a910bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:12.297518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38525f-9345-4cdc-b928-e7ef3a910bc5 None admin] PATH_INFO: `/v3/users/c3d7911f2f5f4f5386cd51a788182aa7` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:12.301512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38525f-9345-4cdc-b928-e7ef3a910bc5 None admin] RBAC: Authorizing `identity:get_user(user_id=c3d7911f2f5f4f5386cd51a788182aa7)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:12.303213 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38525f-9345-4cdc-b928-e7ef3a910bc5 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:12.305268 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 71/145] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:12 2022] GET /identity/v3/users/c3d7911f2f5f4f5386cd51a788182aa7 => generated 289 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:12.310780 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38525f-9345-4cdc-b928-e7ef3a910bc5 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:12.325722 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e999b9-c86f-4410-a4c7-347094582a51 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:12.387631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2757e5b-3387-49ac-b2cc-a1f62fa707e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:12.388370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2757e5b-3387-49ac-b2cc-a1f62fa707e9 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:12.388662 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2757e5b-3387-49ac-b2cc-a1f62fa707e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:12.388947 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2757e5b-3387-49ac-b2cc-a1f62fa707e9 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:12.389679 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2757e5b-3387-49ac-b2cc-a1f62fa707e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:12.391159 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2757e5b-3387-49ac-b2cc-a1f62fa707e9 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:12.428687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 75/146] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:37:12 2022] POST /identity/v3/users => generated 283 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:37:12.514932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38525f-9345-4cdc-b928-e7ef3a910bc5 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:12.569993 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8eddc-f393-495f-85b9-a134625a8f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:12.570822 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8eddc-f393-495f-85b9-a134625a8f20 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:12.571113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8eddc-f393-495f-85b9-a134625a8f20 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:12.571399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8eddc-f393-495f-85b9-a134625a8f20 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:12.572689 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8eddc-f393-495f-85b9-a134625a8f20 None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:12.574259 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8eddc-f393-495f-85b9-a134625a8f20 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:12.576437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 72/147] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:12 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:12.581919 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2757e5b-3387-49ac-b2cc-a1f62fa707e9 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:12.608750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:12.768747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2757e5b-3387-49ac-b2cc-a1f62fa707e9 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:12.805803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121724f9-e3c2-4552-8541-2d4b1be1c4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:12.807051 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121724f9-e3c2-4552-8541-2d4b1be1c4dd None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:12.808078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121724f9-e3c2-4552-8541-2d4b1be1c4dd None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:12.808555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121724f9-e3c2-4552-8541-2d4b1be1c4dd None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82/users/c3d7911f2f5f4f5386cd51a788182aa7/roles/84bd13e4cee04c12bd6a0e549db79246` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:12.810776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121724f9-e3c2-4552-8541-2d4b1be1c4dd None admin] RBAC: Authorizing `identity:create_grant(project_id=ba02b19bd80d44c4a9a9ae6517b11b82, user_id=c3d7911f2f5f4f5386cd51a788182aa7, role_id=84bd13e4cee04c12bd6a0e549db79246)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:12.812536 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121724f9-e3c2-4552-8541-2d4b1be1c4dd None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:12.828993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 76/148] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:37:12 2022] PUT /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82/users/c3d7911f2f5f4f5386cd51a788182aa7/roles/84bd13e4cee04c12bd6a0e549db79246 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:37:12.923959 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2daa36-035b-420b-8c7f-8934f1a2d12f None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:12.924364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2daa36-035b-420b-8c7f-8934f1a2d12f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:12.924560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2daa36-035b-420b-8c7f-8934f1a2d12f None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:12.926007 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 73/149] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:12.932989 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d57c9e-b7a7-4bd7-8b42-792b1b3baddc None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:12.933606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d57c9e-b7a7-4bd7-8b42-792b1b3baddc None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:12.934148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d57c9e-b7a7-4bd7-8b42-792b1b3baddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:12.962751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-97d57c9e-b7a7-4bd7-8b42-792b1b3baddc None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:12.964978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-97d57c9e-b7a7-4bd7-8b42-792b1b3baddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:13.068235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:13.078025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 77/150] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:13.114428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2daa36-035b-420b-8c7f-8934f1a2d12f None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:13.233050 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:13.304926 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2daa36-035b-420b-8c7f-8934f1a2d12f None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:13.341898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3e6d8-b5a1-44f8-8639-95b19bddce71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:13.342713 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3e6d8-b5a1-44f8-8639-95b19bddce71 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:13.342998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3e6d8-b5a1-44f8-8639-95b19bddce71 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:13.343281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3e6d8-b5a1-44f8-8639-95b19bddce71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:13.355072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3e6d8-b5a1-44f8-8639-95b19bddce71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-7ae3e6d8-b5a1-44f8-8639-95b19bddce71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:13.363130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:13.364192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:13.365134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:13.365134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:13.365134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:13.365134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:13.365134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:13.365134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:13.365134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:13.365134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:13.365134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:13.365134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:13.365134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:13.365134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:13.365134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 74/151] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:37:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:13.371990 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d57c9e-b7a7-4bd7-8b42-792b1b3baddc None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:13.500762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d57c9e-b7a7-4bd7-8b42-792b1b3baddc None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:13.548702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b7e6e-d138-4ca6-9ca6-aa96f06c8022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:13.549468 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7e6e-d138-4ca6-9ca6-aa96f06c8022 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:13.549760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7e6e-d138-4ca6-9ca6-aa96f06c8022 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:13.550042 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b7e6e-d138-4ca6-9ca6-aa96f06c8022 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:13.550871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b7e6e-d138-4ca6-9ca6-aa96f06c8022 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:13.551187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b7e6e-d138-4ca6-9ca6-aa96f06c8022 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:13.552619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b7e6e-d138-4ca6-9ca6-aa96f06c8022 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:13.562369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 78/152] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:13.568008 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3e6d8-b5a1-44f8-8639-95b19bddce71 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:13.696491 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3e6d8-b5a1-44f8-8639-95b19bddce71 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:13.712208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:13.743993 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae33ec2-98bc-4356-be43-6def3b557206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:13.744898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae33ec2-98bc-4356-be43-6def3b557206 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:13.745218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae33ec2-98bc-4356-be43-6def3b557206 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:13.745500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae33ec2-98bc-4356-be43-6def3b557206 None admin] PATH_INFO: `/v3/users/7d77dedfd6bd435d94f994cfcde5100a` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:13.748717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae33ec2-98bc-4356-be43-6def3b557206 None admin] RBAC: Authorizing `identity:get_user(user_id=7d77dedfd6bd435d94f994cfcde5100a)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:13.750197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae33ec2-98bc-4356-be43-6def3b557206 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:13.752251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 75/153] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:13 2022] GET /identity/v3/users/7d77dedfd6bd435d94f994cfcde5100a => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:13.757454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b7e6e-d138-4ca6-9ca6-aa96f06c8022 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:13.886435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:13.933366 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b7e6e-d138-4ca6-9ca6-aa96f06c8022 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:13.969978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4d6be-eb54-4b84-9451-813ac14e9e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:13.970575 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4d6be-eb54-4b84-9451-813ac14e9e9e None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:13.970853 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4d6be-eb54-4b84-9451-813ac14e9e9e None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:13.971130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4d6be-eb54-4b84-9451-813ac14e9e9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:14.052942 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4d6be-eb54-4b84-9451-813ac14e9e9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:14.054433 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4d6be-eb54-4b84-9451-813ac14e9e9e None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:14.128853 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:14.177856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 79/154] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:14.184034 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae33ec2-98bc-4356-be43-6def3b557206 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:14.288279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d20500-cd70-45ac-aed2-f305ef1ca71c None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:14.288563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d20500-cd70-45ac-aed2-f305ef1ca71c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:14.288748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d20500-cd70-45ac-aed2-f305ef1ca71c None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:14.290854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 80/155] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:37:14.300282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45efaa6-22bc-43e7-801a-d7dd30140177 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:14.300974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45efaa6-22bc-43e7-801a-d7dd30140177 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:14.301257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45efaa6-22bc-43e7-801a-d7dd30140177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:14.316689 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae33ec2-98bc-4356-be43-6def3b557206 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:14.335828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-d45efaa6-22bc-43e7-801a-d7dd30140177 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:14.337235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d45efaa6-22bc-43e7-801a-d7dd30140177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:14.451387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f4f1d-e1b3-4f87-96b3-53edb79a5612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:14.452743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f4f1d-e1b3-4f87-96b3-53edb79a5612 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:14.453245 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f4f1d-e1b3-4f87-96b3-53edb79a5612 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:14.453727 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f4f1d-e1b3-4f87-96b3-53edb79a5612 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:14.458848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 81/156] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:14.459872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f4f1d-e1b3-4f87-96b3-53edb79a5612 None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:14.461605 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f4f1d-e1b3-4f87-96b3-53edb79a5612 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:14.463627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 76/157] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:14 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:14.470215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45efaa6-22bc-43e7-801a-d7dd30140177 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:14.502183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f4f1d-e1b3-4f87-96b3-53edb79a5612 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:14.589162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:14.601606 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:14.605126 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45efaa6-22bc-43e7-801a-d7dd30140177 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:14.658385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413282f-4a66-4278-b147-415431cbde0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:14.659059 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413282f-4a66-4278-b147-415431cbde0f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:14.659342 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413282f-4a66-4278-b147-415431cbde0f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:14.659618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e413282f-4a66-4278-b147-415431cbde0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:14.661430 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413282f-4a66-4278-b147-415431cbde0f None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82, user.id=7d77dedfd6bd435d94f994cfcde5100a) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:14.661749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413282f-4a66-4278-b147-415431cbde0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:14.663229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e413282f-4a66-4278-b147-415431cbde0f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:14.674930 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 82/158] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:14 2022] GET /identity/v3/role_assignments?user.id=7d77dedfd6bd435d94f994cfcde5100a&scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:14.707313 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f4f1d-e1b3-4f87-96b3-53edb79a5612 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:14.758164 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db601cc-f512-4662-b9b2-55a16d148313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:14.759324 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db601cc-f512-4662-b9b2-55a16d148313 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:14.759810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db601cc-f512-4662-b9b2-55a16d148313 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:14.760326 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db601cc-f512-4662-b9b2-55a16d148313 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:14.775174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db601cc-f512-4662-b9b2-55a16d148313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-8db601cc-f512-4662-b9b2-55a16d148313 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:14.783628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:14.784657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:14.785431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:14.785431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:14.785431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:14.785431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:14.785431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:14.785431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:14.785431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:14.785431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:14.785431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:14.785431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:14.785431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:14.785431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:14.787864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 77/159] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:14.794549 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413282f-4a66-4278-b147-415431cbde0f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:14.926643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e413282f-4a66-4278-b147-415431cbde0f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:14.965347 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a62bc-9d91-4996-bde4-edcb52a885a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:14.966033 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a62bc-9d91-4996-bde4-edcb52a885a9 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:14.966407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a62bc-9d91-4996-bde4-edcb52a885a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:14.966576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97a62bc-9d91-4996-bde4-edcb52a885a9 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:14.967328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a62bc-9d91-4996-bde4-edcb52a885a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:14.967627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a62bc-9d91-4996-bde4-edcb52a885a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:14.969042 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97a62bc-9d91-4996-bde4-edcb52a885a9 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:14.978374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 83/160] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:14.984476 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db601cc-f512-4662-b9b2-55a16d148313 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:15.126335 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db601cc-f512-4662-b9b2-55a16d148313 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:15.129668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:15.165285 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f8cc8-6592-4c1d-aa87-19523a25e33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:15.166537 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f8cc8-6592-4c1d-aa87-19523a25e33c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:15.167121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f8cc8-6592-4c1d-aa87-19523a25e33c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:15.167631 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f8cc8-6592-4c1d-aa87-19523a25e33c None admin] PATH_INFO: `/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:15.172672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f8cc8-6592-4c1d-aa87-19523a25e33c None admin] RBAC: Authorizing `identity:get_user(user_id=7db339453aae4a3fa6c2ae5cb3a3a2eb)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:15.174382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4f8cc8-6592-4c1d-aa87-19523a25e33c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:15.176342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 78/161] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:14 2022] GET /identity/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:15.182169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a62bc-9d91-4996-bde4-edcb52a885a9 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:15.215611 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:15.317632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97a62bc-9d91-4996-bde4-edcb52a885a9 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:15.340548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb816138-809b-4709-98fe-1be36f37b99d None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:15.341579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb816138-809b-4709-98fe-1be36f37b99d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:15.342202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb816138-809b-4709-98fe-1be36f37b99d None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:15.344049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 79/162] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:15.350928 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd98b8-97e5-4870-be7a-7fc2610d415a None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:15.351512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd98b8-97e5-4870-be7a-7fc2610d415a None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:15.352030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddd98b8-97e5-4870-be7a-7fc2610d415a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:15.371533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33606c-47d3-4fb7-87a2-650161a49c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:15.372229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33606c-47d3-4fb7-87a2-650161a49c07 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:15.372513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33606c-47d3-4fb7-87a2-650161a49c07 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:15.372787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec33606c-47d3-4fb7-87a2-650161a49c07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:15.389120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5ddd98b8-97e5-4870-be7a-7fc2610d415a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:15.390847 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5ddd98b8-97e5-4870-be7a-7fc2610d415a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:15.509062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33606c-47d3-4fb7-87a2-650161a49c07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:15.510571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec33606c-47d3-4fb7-87a2-650161a49c07 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:15.538942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 80/163] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:15.540791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f1501-44cb-429b-8084-0527dedeb079 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:15.541315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f1501-44cb-429b-8084-0527dedeb079 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:15.541800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f1501-44cb-429b-8084-0527dedeb079 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:15.544161 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 81/164] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:15.549934 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:15.551578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ef6bd-d2e1-479c-a286-544768c20fdd None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:15.552330 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ef6bd-d2e1-479c-a286-544768c20fdd None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:15.552900 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ef6bd-d2e1-479c-a286-544768c20fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:15.583939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-1f0ef6bd-d2e1-479c-a286-544768c20fdd None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:15.585479 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1f0ef6bd-d2e1-479c-a286-544768c20fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:15.714637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 82/165] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:15.716305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ef6bd-d2e1-479c-a286-544768c20fdd None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:15.750652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 84/166] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:15.756367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33606c-47d3-4fb7-87a2-650161a49c07 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:15.794045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:15.967626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec33606c-47d3-4fb7-87a2-650161a49c07 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:15.980326 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ef6bd-d2e1-479c-a286-544768c20fdd None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:16.016293 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:16.039295 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9cee7-f4d6-494b-9041-8c4cbffcb251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:16.040147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9cee7-f4d6-494b-9041-8c4cbffcb251 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:16.040429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9cee7-f4d6-494b-9041-8c4cbffcb251 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:16.040701 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9cee7-f4d6-494b-9041-8c4cbffcb251 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:16.060089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9cee7-f4d6-494b-9041-8c4cbffcb251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:16.061769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a619609-3677-4e9c-8536-46ff002c8629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:16.062663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a619609-3677-4e9c-8536-46ff002c8629 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:16.062950 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a619609-3677-4e9c-8536-46ff002c8629 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:16.063232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a619609-3677-4e9c-8536-46ff002c8629 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-64c9cee7-f4d6-494b-9041-8c4cbffcb251 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:16.068309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:16.069258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:16.070187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:16.070187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:16.070187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:16.070187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:16.070187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:16.070187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:16.070187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:16.070187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:16.070187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:16.070187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:16.070187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:16.070187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:16.070788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a619609-3677-4e9c-8536-46ff002c8629 None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:16.070864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 83/167] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:37:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:16.070936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a619609-3677-4e9c-8536-46ff002c8629 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:16.071007 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9cee7-f4d6-494b-9041-8c4cbffcb251 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:16.072148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 85/168] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:15 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:16.077990 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a619609-3677-4e9c-8536-46ff002c8629 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:16.203681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:16.221404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9cee7-f4d6-494b-9041-8c4cbffcb251 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:16.275187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a619609-3677-4e9c-8536-46ff002c8629 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:16.279252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1282faf-b1ef-4672-9ff1-0d0bc19e3f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:16.280094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1282faf-b1ef-4672-9ff1-0d0bc19e3f40 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:16.280376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1282faf-b1ef-4672-9ff1-0d0bc19e3f40 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:16.280647 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1282faf-b1ef-4672-9ff1-0d0bc19e3f40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:16.298301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1282faf-b1ef-4672-9ff1-0d0bc19e3f40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-e1282faf-b1ef-4672-9ff1-0d0bc19e3f40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:16.314634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:16.315543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:16.316359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:16.316359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:16.316359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:16.316359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:16.316359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:16.316359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:16.316359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:16.316359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:16.316359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:16.316359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:16.316359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:16.316359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:16.316359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 84/169] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:16.317021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1282faf-b1ef-4672-9ff1-0d0bc19e3f40 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:16.364726 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f800b6-fe82-4ab5-b13c-6f4689d76778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:16.365442 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f800b6-fe82-4ab5-b13c-6f4689d76778 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:16.365730 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f800b6-fe82-4ab5-b13c-6f4689d76778 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:16.366013 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f800b6-fe82-4ab5-b13c-6f4689d76778 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:16.366790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f800b6-fe82-4ab5-b13c-6f4689d76778 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:16.367103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f800b6-fe82-4ab5-b13c-6f4689d76778 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:16.368544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f800b6-fe82-4ab5-b13c-6f4689d76778 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:16.378466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 86/170] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:37:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:16.384011 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f800b6-fe82-4ab5-b13c-6f4689d76778 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:16.435456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:16.484581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1282faf-b1ef-4672-9ff1-0d0bc19e3f40 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:16.525297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f82a6-3070-41b1-96c7-362985ac795e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:16.526888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f82a6-3070-41b1-96c7-362985ac795e None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:16.527431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f82a6-3070-41b1-96c7-362985ac795e None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:16.527907 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f82a6-3070-41b1-96c7-362985ac795e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:16.529629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f82a6-3070-41b1-96c7-362985ac795e None admin] RBAC: Adding query filter params (role.id=fbd0f348273f4791b1813d1df24df10e, scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82, user.id=7db339453aae4a3fa6c2ae5cb3a3a2eb) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:16.530173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f82a6-3070-41b1-96c7-362985ac795e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:16.531838 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f82a6-3070-41b1-96c7-362985ac795e None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:16.540834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 85/171] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:16 2022] GET /identity/v3/role_assignments?user.id=7db339453aae4a3fa6c2ae5cb3a3a2eb&scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82&role.id=fbd0f348273f4791b1813d1df24df10e => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:16.544431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f82a6-3070-41b1-96c7-362985ac795e None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:16.564257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f800b6-fe82-4ab5-b13c-6f4689d76778 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:16.598409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:16.656895 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ff170-0a67-4c78-8cd4-ce9fa18c2c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:16.657973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ff170-0a67-4c78-8cd4-ce9fa18c2c92 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:16.658548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ff170-0a67-4c78-8cd4-ce9fa18c2c92 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:16.659192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ff170-0a67-4c78-8cd4-ce9fa18c2c92 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:16.661465 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ff170-0a67-4c78-8cd4-ce9fa18c2c92 None admin] RBAC: Adding query filter params (name=member) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:16.662370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ff170-0a67-4c78-8cd4-ce9fa18c2c92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:16.665226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ff170-0a67-4c78-8cd4-ce9fa18c2c92 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:16.686161 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 87/172] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:16.688139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704dba2-b08f-4646-bcae-9156404d60df None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:16.689828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704dba2-b08f-4646-bcae-9156404d60df None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:16.690495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1704dba2-b08f-4646-bcae-9156404d60df None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:16.693601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 88/173] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:16.695715 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704dba2-b08f-4646-bcae-9156404d60df None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:16.791691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f82a6-3070-41b1-96c7-362985ac795e None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:16.833174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1704dba2-b08f-4646-bcae-9156404d60df None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:16.838429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145e34a-1386-4288-992a-cf3c9baa4192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:16.839367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145e34a-1386-4288-992a-cf3c9baa4192 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:16.839656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145e34a-1386-4288-992a-cf3c9baa4192 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:16.839968 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145e34a-1386-4288-992a-cf3c9baa4192 None admin] PATH_INFO: `/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:16.841224 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145e34a-1386-4288-992a-cf3c9baa4192 None admin] RBAC: Authorizing `identity:get_user(user_id=7db339453aae4a3fa6c2ae5cb3a3a2eb)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:16.842704 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145e34a-1386-4288-992a-cf3c9baa4192 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:16.845110 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 86/174] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:16 2022] GET /identity/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb => generated 275 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:16.846077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:16.846397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126965b-c383-454e-90da-63d2c09a7d69 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:16.846691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126965b-c383-454e-90da-63d2c09a7d69 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:16.846977 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2126965b-c383-454e-90da-63d2c09a7d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:16.867236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-2126965b-c383-454e-90da-63d2c09a7d69 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:16.869257 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-2126965b-c383-454e-90da-63d2c09a7d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:16.887814 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217bf72c-e30c-44eb-9522-68d01850e778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:16.888775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bf72c-e30c-44eb-9522-68d01850e778 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:16.889336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bf72c-e30c-44eb-9522-68d01850e778 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:16.889688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bf72c-e30c-44eb-9522-68d01850e778 None admin] PATH_INFO: `/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:16.890955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217bf72c-e30c-44eb-9522-68d01850e778 None admin] RBAC: Authorizing `identity:get_user(user_id=7db339453aae4a3fa6c2ae5cb3a3a2eb)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:16.892959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217bf72c-e30c-44eb-9522-68d01850e778 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:16.894657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 89/175] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:16 2022] GET /identity/v3/users/7db339453aae4a3fa6c2ae5cb3a3a2eb => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:16.895892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcd93d-5266-4661-8c38-796ac9a7f4ba None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:16.896747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcd93d-5266-4661-8c38-796ac9a7f4ba None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:16.897051 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbcd93d-5266-4661-8c38-796ac9a7f4ba None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:16.898776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 90/176] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:16.899582 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcd93d-5266-4661-8c38-796ac9a7f4ba None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:16.956732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 87/177] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:16.957951 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126965b-c383-454e-90da-63d2c09a7d69 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:16.992113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:17.068941 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbcd93d-5266-4661-8c38-796ac9a7f4ba None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:17.108081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2126965b-c383-454e-90da-63d2c09a7d69 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:17.113946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1823-dce1-44f0-9514-8472bffbc0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:17.121048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1823-dce1-44f0-9514-8472bffbc0e4 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:17.121582 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1823-dce1-44f0-9514-8472bffbc0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:17.122146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1823-dce1-44f0-9514-8472bffbc0e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:17.153263 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08a3d0-7da1-43ec-b7eb-c6bd14b837c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:17.153960 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08a3d0-7da1-43ec-b7eb-c6bd14b837c0 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:17.154247 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08a3d0-7da1-43ec-b7eb-c6bd14b837c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:17.154529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08a3d0-7da1-43ec-b7eb-c6bd14b837c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:17.180483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:17.215142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f1823-dce1-44f0-9514-8472bffbc0e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:17.217816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f1823-dce1-44f0-9514-8472bffbc0e4 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:17.248419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08a3d0-7da1-43ec-b7eb-c6bd14b837c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:17.250227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08a3d0-7da1-43ec-b7eb-c6bd14b837c0 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:17.269203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:17.350431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 91/178] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:17.351752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46734b3f-2cb3-4bbe-a422-2a2ae18ed3f1 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:17.352074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46734b3f-2cb3-4bbe-a422-2a2ae18ed3f1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:17.352356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46734b3f-2cb3-4bbe-a422-2a2ae18ed3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:17.370676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-46734b3f-2cb3-4bbe-a422-2a2ae18ed3f1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:17.372263 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-46734b3f-2cb3-4bbe-a422-2a2ae18ed3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:17.379812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 88/179] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:17.379812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08a3d0-7da1-43ec-b7eb-c6bd14b837c0 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:17.450420 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 92/180] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:17.451800 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4d191-be7b-4d93-9e1c-b98d0986ef64 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:17.452125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4d191-be7b-4d93-9e1c-b98d0986ef64 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:17.452404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4d191-be7b-4d93-9e1c-b98d0986ef64 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:17.457967 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 93/181] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:17.457967 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4d191-be7b-4d93-9e1c-b98d0986ef64 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:17.472304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:17.500598 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08a3d0-7da1-43ec-b7eb-c6bd14b837c0 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:17.516067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:17.538768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06ea43-4907-4e4f-bcd0-2979a58c43d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:17.539611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06ea43-4907-4e4f-bcd0-2979a58c43d1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:17.539902 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06ea43-4907-4e4f-bcd0-2979a58c43d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:17.540220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06ea43-4907-4e4f-bcd0-2979a58c43d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:17.552177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06ea43-4907-4e4f-bcd0-2979a58c43d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-fe06ea43-4907-4e4f-bcd0-2979a58c43d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:17.560607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:17.561610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:17.562391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:17.562391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:17.562391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:17.562391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:17.562391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:17.562391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:17.562391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:17.562391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:17.562391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:17.562391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:17.562391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:17.562391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:17.562391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 89/182] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:37:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:17.563222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06ea43-4907-4e4f-bcd0-2979a58c43d1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:17.579546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d4d191-be7b-4d93-9e1c-b98d0986ef64 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:17.615133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a547a6d-36c6-4f1f-b5ea-e69a856bc675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:17.616040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a547a6d-36c6-4f1f-b5ea-e69a856bc675 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:17.616330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a547a6d-36c6-4f1f-b5ea-e69a856bc675 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:17.616617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a547a6d-36c6-4f1f-b5ea-e69a856bc675 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:17.617822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a547a6d-36c6-4f1f-b5ea-e69a856bc675 None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:17.619550 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a547a6d-36c6-4f1f-b5ea-e69a856bc675 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:17.622165 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 94/183] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:17 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:17.623274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40291155-8ff6-4607-b591-622a4a0338ef None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:17.623571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40291155-8ff6-4607-b591-622a4a0338ef None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:17.623851 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40291155-8ff6-4607-b591-622a4a0338ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:17.640371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-40291155-8ff6-4607-b591-622a4a0338ef None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:17.641568 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-40291155-8ff6-4607-b591-622a4a0338ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:17.666784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06ea43-4907-4e4f-bcd0-2979a58c43d1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:17.700379 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e25bb3-c354-48f1-bf1b-27a8936596b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:17.701258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e25bb3-c354-48f1-bf1b-27a8936596b4 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:17.701544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e25bb3-c354-48f1-bf1b-27a8936596b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:17.701825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e25bb3-c354-48f1-bf1b-27a8936596b4 None admin] PATH_INFO: `/v3/projects/8994e6acc2c242949ceaf4437c49317e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:17.703019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e25bb3-c354-48f1-bf1b-27a8936596b4 None admin] RBAC: Authorizing `identity:get_project(project_id=8994e6acc2c242949ceaf4437c49317e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:17.704523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e25bb3-c354-48f1-bf1b-27a8936596b4 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:17.706220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 90/184] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:17 2022] GET /identity/v3/projects/8994e6acc2c242949ceaf4437c49317e => generated 312 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:17.706505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 95/185] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:17.706989 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e25bb3-c354-48f1-bf1b-27a8936596b4 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:17.707811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62522778-ccaa-45f3-b289-a4933a93f869 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:17.708129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62522778-ccaa-45f3-b289-a4933a93f869 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:17.708408 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62522778-ccaa-45f3-b289-a4933a93f869 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:17.709883 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 96/186] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:17.720615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62522778-ccaa-45f3-b289-a4933a93f869 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:17.722111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:17.753193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:17.808888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e25bb3-c354-48f1-bf1b-27a8936596b4 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:17.819164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62522778-ccaa-45f3-b289-a4933a93f869 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:17.840485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec647a-6a82-454b-80c3-2352e20924a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:17.841271 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec647a-6a82-454b-80c3-2352e20924a9 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:17.841554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec647a-6a82-454b-80c3-2352e20924a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:17.841824 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec647a-6a82-454b-80c3-2352e20924a9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:17.851510 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576f4f6-7d1d-4f6d-9729-ca4d3bb9eca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:17.852207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576f4f6-7d1d-4f6d-9729-ca4d3bb9eca0 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:17.852337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec647a-6a82-454b-80c3-2352e20924a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:17.852488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576f4f6-7d1d-4f6d-9729-ca4d3bb9eca0 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:17.852762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6576f4f6-7d1d-4f6d-9729-ca4d3bb9eca0 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:17.853503 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576f4f6-7d1d-4f6d-9729-ca4d3bb9eca0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:17.853810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576f4f6-7d1d-4f6d-9729-ca4d3bb9eca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:17.855694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6576f4f6-7d1d-4f6d-9729-ca4d3bb9eca0 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-afec647a-6a82-454b-80c3-2352e20924a9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:17.860505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:17.861581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:17.862406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:17.862406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:17.862406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:17.862406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:17.862406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:17.862406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:17.862406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:17.862406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:17.862406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:17.862406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:17.862406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:17.862406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:17.862406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 91/187] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:17.863053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec647a-6a82-454b-80c3-2352e20924a9 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:17.871792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 97/188] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:17.871792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576f4f6-7d1d-4f6d-9729-ca4d3bb9eca0 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:17.935112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:17.963068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec647a-6a82-454b-80c3-2352e20924a9 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:17.964640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:17.971147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6576f4f6-7d1d-4f6d-9729-ca4d3bb9eca0 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:17.998431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf935f-e18c-402a-aaf2-32e7354f21a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:17.999087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf935f-e18c-402a-aaf2-32e7354f21a3 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:17.999381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf935f-e18c-402a-aaf2-32e7354f21a3 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:17.999657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf935f-e18c-402a-aaf2-32e7354f21a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:18.000548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf935f-e18c-402a-aaf2-32e7354f21a3 None admin] RBAC: Adding query filter params (role.id=8427479f632a4da3949971f58b438c7f, scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82, user.id=7db339453aae4a3fa6c2ae5cb3a3a2eb) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:18.000860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf935f-e18c-402a-aaf2-32e7354f21a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:18.002311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf935f-e18c-402a-aaf2-32e7354f21a3 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:18.004807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55e146-0059-4cea-b108-fcfe40aa8ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:18.005476 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e146-0059-4cea-b108-fcfe40aa8ad6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:18.005762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e146-0059-4cea-b108-fcfe40aa8ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:18.006041 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af55e146-0059-4cea-b108-fcfe40aa8ad6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:18.006879 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55e146-0059-4cea-b108-fcfe40aa8ad6 None admin] RBAC: Adding query filter params (role.id=fbd0f348273f4791b1813d1df24df10e, scope.project.id=8994e6acc2c242949ceaf4437c49317e, user.id=7db339453aae4a3fa6c2ae5cb3a3a2eb) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:18.007194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55e146-0059-4cea-b108-fcfe40aa8ad6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:18.008176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 92/189] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:17 2022] GET /identity/v3/role_assignments?user.id=7db339453aae4a3fa6c2ae5cb3a3a2eb&scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82&role.id=8427479f632a4da3949971f58b438c7f => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:18.008670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af55e146-0059-4cea-b108-fcfe40aa8ad6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:18.009369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1912374-c217-48e0-bc76-a6657ef79e2a None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:18.009661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1912374-c217-48e0-bc76-a6657ef79e2a None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:18.009940 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1912374-c217-48e0-bc76-a6657ef79e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:18.016176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 98/190] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:17 2022] GET /identity/v3/role_assignments?user.id=7db339453aae4a3fa6c2ae5cb3a3a2eb&scope.project.id=8994e6acc2c242949ceaf4437c49317e&role.id=fbd0f348273f4791b1813d1df24df10e => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:18.016976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55e146-0059-4cea-b108-fcfe40aa8ad6 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:18.038182 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-b1912374-c217-48e0-bc76-a6657ef79e2a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:18.039862 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b1912374-c217-48e0-bc76-a6657ef79e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:18.136842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 93/191] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:18.138322 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1912374-c217-48e0-bc76-a6657ef79e2a None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:18.143106 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af55e146-0059-4cea-b108-fcfe40aa8ad6 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:18.188545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:18.196727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d24f6-e004-4809-9e9a-ed018886bb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:18.197810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d24f6-e004-4809-9e9a-ed018886bb00 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:18.198223 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d24f6-e004-4809-9e9a-ed018886bb00 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:18.199544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d24f6-e004-4809-9e9a-ed018886bb00 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:18.200653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d24f6-e004-4809-9e9a-ed018886bb00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:18.202247 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d24f6-e004-4809-9e9a-ed018886bb00 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:18.204406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 99/192] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:37:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:18.205214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d24f6-e004-4809-9e9a-ed018886bb00 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:18.287819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:18.299271 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1912374-c217-48e0-bc76-a6657ef79e2a None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:18.333365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d24f6-e004-4809-9e9a-ed018886bb00 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:18.334963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ff2c0-9c4d-46a2-9e6b-0b1356d72ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:18.335638 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ff2c0-9c4d-46a2-9e6b-0b1356d72ee6 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:18.335915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ff2c0-9c4d-46a2-9e6b-0b1356d72ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:18.336241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281ff2c0-9c4d-46a2-9e6b-0b1356d72ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:18.336998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ff2c0-9c4d-46a2-9e6b-0b1356d72ee6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:18.337303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ff2c0-9c4d-46a2-9e6b-0b1356d72ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:18.338705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281ff2c0-9c4d-46a2-9e6b-0b1356d72ee6 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:18.347663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 94/193] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:18.348517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ff2c0-9c4d-46a2-9e6b-0b1356d72ee6 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:18.374228 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44547d41-f2aa-422a-a999-0d8c9642dcde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:18.375107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44547d41-f2aa-422a-a999-0d8c9642dcde None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:18.375392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44547d41-f2aa-422a-a999-0d8c9642dcde None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:18.375696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44547d41-f2aa-422a-a999-0d8c9642dcde None admin] PATH_INFO: `/v3/users/38cd7f6cad2e43158de48fde6325e0f9` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:18.389606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44547d41-f2aa-422a-a999-0d8c9642dcde None admin] RBAC: Authorizing `identity:get_user(user_id=38cd7f6cad2e43158de48fde6325e0f9)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:18.391081 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44547d41-f2aa-422a-a999-0d8c9642dcde None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:18.392839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 100/194] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:18 2022] GET /identity/v3/users/38cd7f6cad2e43158de48fde6325e0f9 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:18.397346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44547d41-f2aa-422a-a999-0d8c9642dcde None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:18.450175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:18.454462 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281ff2c0-9c4d-46a2-9e6b-0b1356d72ee6 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:18.487689 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65007b-cc26-4849-9524-f9672533f3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:18.488528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65007b-cc26-4849-9524-f9672533f3bd None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:18.488814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65007b-cc26-4849-9524-f9672533f3bd None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:18.489090 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65007b-cc26-4849-9524-f9672533f3bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:18.493065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44547d41-f2aa-422a-a999-0d8c9642dcde None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:18.499952 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca65007b-cc26-4849-9524-f9672533f3bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-ca65007b-cc26-4849-9524-f9672533f3bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:18.508129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:18.509672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:18.510625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:18.510625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:18.510625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:18.510625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:18.510625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:18.510625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:18.510625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:18.510625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:18.510625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:18.510625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:18.510625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:18.510625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:18.510625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 95/195] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:37:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:18.510625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65007b-cc26-4849-9524-f9672533f3bd None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:18.530554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:18.533165 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e9b5a-09d4-472b-97dc-d96ce7933ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:18.533856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e9b5a-09d4-472b-97dc-d96ce7933ee4 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:18.534137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e9b5a-09d4-472b-97dc-d96ce7933ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:18.534412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e9b5a-09d4-472b-97dc-d96ce7933ee4 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:18.535131 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e9b5a-09d4-472b-97dc-d96ce7933ee4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:18.536619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e9b5a-09d4-472b-97dc-d96ce7933ee4 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:18.556152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 101/196] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:37:18 2022] POST /identity/v3/users => generated 297 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:37:18.565107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e9b5a-09d4-472b-97dc-d96ce7933ee4 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:18.628766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65007b-cc26-4849-9524-f9672533f3bd None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:18.668069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161c8c4-7e04-48fa-8deb-93cbe2a8c756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:18.668964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161c8c4-7e04-48fa-8deb-93cbe2a8c756 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:18.669242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161c8c4-7e04-48fa-8deb-93cbe2a8c756 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:18.669517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4161c8c4-7e04-48fa-8deb-93cbe2a8c756 None admin] PATH_INFO: `/v3/users/c3d7911f2f5f4f5386cd51a788182aa7` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:18.670834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161c8c4-7e04-48fa-8deb-93cbe2a8c756 None admin] RBAC: Authorizing `identity:get_user(user_id=c3d7911f2f5f4f5386cd51a788182aa7)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:18.672351 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4161c8c4-7e04-48fa-8deb-93cbe2a8c756 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:18.674296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 96/197] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:18 2022] GET /identity/v3/users/c3d7911f2f5f4f5386cd51a788182aa7 => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:18.674926 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161c8c4-7e04-48fa-8deb-93cbe2a8c756 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:18.681835 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e9b5a-09d4-472b-97dc-d96ce7933ee4 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:18.696923 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:18.728865 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5e1a3-1303-45e0-b53c-6cf36195f657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:18.730443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5e1a3-1303-45e0-b53c-6cf36195f657 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:18.730953 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5e1a3-1303-45e0-b53c-6cf36195f657 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:18.731420 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5e1a3-1303-45e0-b53c-6cf36195f657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:18.783825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:18.802725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4161c8c4-7e04-48fa-8deb-93cbe2a8c756 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:18.811670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5e1a3-1303-45e0-b53c-6cf36195f657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:18.813226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf5e1a3-1303-45e0-b53c-6cf36195f657 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:18.843441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590001d-f175-485a-9870-70a4273e6fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:18.844259 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590001d-f175-485a-9870-70a4273e6fc1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:18.844543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590001d-f175-485a-9870-70a4273e6fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:18.844819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590001d-f175-485a-9870-70a4273e6fc1 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:18.845599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590001d-f175-485a-9870-70a4273e6fc1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:18.845880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590001d-f175-485a-9870-70a4273e6fc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:18.847264 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590001d-f175-485a-9870-70a4273e6fc1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:18.857330 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 97/198] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:37:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:18.857527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590001d-f175-485a-9870-70a4273e6fc1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:18.946353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 102/199] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:18.947240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5e1a3-1303-45e0-b53c-6cf36195f657 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:18.966595 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:18.997159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590001d-f175-485a-9870-70a4273e6fc1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:19.036112 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e10268-1c95-4ba5-954d-71b51e09fb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:19.037219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e10268-1c95-4ba5-954d-71b51e09fb4f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:19.037753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e10268-1c95-4ba5-954d-71b51e09fb4f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:19.038288 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e10268-1c95-4ba5-954d-71b51e09fb4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:19.056579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf5e1a3-1303-45e0-b53c-6cf36195f657 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:19.059383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:19.095702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31319f3-aa8e-4255-9fcc-442d9aea8545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:19.096667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31319f3-aa8e-4255-9fcc-442d9aea8545 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:19.096948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31319f3-aa8e-4255-9fcc-442d9aea8545 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:19.097222 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31319f3-aa8e-4255-9fcc-442d9aea8545 None admin] PATH_INFO: `/v3/users/38cd7f6cad2e43158de48fde6325e0f9` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:19.098518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31319f3-aa8e-4255-9fcc-442d9aea8545 None admin] RBAC: Authorizing `identity:get_user(user_id=38cd7f6cad2e43158de48fde6325e0f9)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:19.100082 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31319f3-aa8e-4255-9fcc-442d9aea8545 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:19.102084 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 103/200] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:18 2022] GET /identity/v3/users/38cd7f6cad2e43158de48fde6325e0f9 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:19.102932 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31319f3-aa8e-4255-9fcc-442d9aea8545 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:19.132652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e10268-1c95-4ba5-954d-71b51e09fb4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:19.134569 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e10268-1c95-4ba5-954d-71b51e09fb4f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:19.187943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:19.213094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31319f3-aa8e-4255-9fcc-442d9aea8545 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:19.245854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 98/201] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:19.247079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e10268-1c95-4ba5-954d-71b51e09fb4f None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:19.260264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e9986-2ea3-404c-a5d1-82e56ca924a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:19.262041 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e9986-2ea3-404c-a5d1-82e56ca924a4 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:19.262325 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e9986-2ea3-404c-a5d1-82e56ca924a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:19.262598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e9986-2ea3-404c-a5d1-82e56ca924a4 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:19.264040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e9986-2ea3-404c-a5d1-82e56ca924a4 None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:19.265773 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e9986-2ea3-404c-a5d1-82e56ca924a4 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:19.271958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 104/202] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:19 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:19.271958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e9986-2ea3-404c-a5d1-82e56ca924a4 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:19.315468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:19.383075 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e9986-2ea3-404c-a5d1-82e56ca924a4 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:19.386438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e10268-1c95-4ba5-954d-71b51e09fb4f None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:19.426215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cedf8-14f7-48ca-9c93-0e4ab844b46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:19.427451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cedf8-14f7-48ca-9c93-0e4ab844b46b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:19.428001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cedf8-14f7-48ca-9c93-0e4ab844b46b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:19.428482 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cedf8-14f7-48ca-9c93-0e4ab844b46b None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:19.429944 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816cedf8-14f7-48ca-9c93-0e4ab844b46b None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:19.431730 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816cedf8-14f7-48ca-9c93-0e4ab844b46b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:19.434518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 105/203] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:19 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:19.434981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a09a1-fb2b-45d8-a071-2d813ae062c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:19.435633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a09a1-fb2b-45d8-a071-2d813ae062c8 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:19.435935 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a09a1-fb2b-45d8-a071-2d813ae062c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:19.436236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a09a1-fb2b-45d8-a071-2d813ae062c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:19.436527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cedf8-14f7-48ca-9c93-0e4ab844b46b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:19.457090 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:19.530157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a09a1-fb2b-45d8-a071-2d813ae062c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:19.531663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1a09a1-fb2b-45d8-a071-2d813ae062c8 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:19.550928 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cedf8-14f7-48ca-9c93-0e4ab844b46b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:19.582645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:19.587966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a505c-bb8f-41db-9d70-11f387f7948a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:19.589714 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a505c-bb8f-41db-9d70-11f387f7948a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:19.590527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a505c-bb8f-41db-9d70-11f387f7948a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:19.591062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a505c-bb8f-41db-9d70-11f387f7948a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:19.592148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a505c-bb8f-41db-9d70-11f387f7948a None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2, user.id=38cd7f6cad2e43158de48fde6325e0f9) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:19.592676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a505c-bb8f-41db-9d70-11f387f7948a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:19.594335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a505c-bb8f-41db-9d70-11f387f7948a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:19.600696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 106/204] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:19 2022] GET /identity/v3/role_assignments?user.id=38cd7f6cad2e43158de48fde6325e0f9&scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:19.609218 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a505c-bb8f-41db-9d70-11f387f7948a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:19.640516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 99/205] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:19.645929 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a09a1-fb2b-45d8-a071-2d813ae062c8 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:19.760651 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:19.763806 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1a09a1-fb2b-45d8-a071-2d813ae062c8 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:19.802640 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a505c-bb8f-41db-9d70-11f387f7948a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:19.803546 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b6ec6-6727-42e4-82be-873d532ea659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:19.804451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b6ec6-6727-42e4-82be-873d532ea659 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:19.804734 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b6ec6-6727-42e4-82be-873d532ea659 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:19.805008 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b6ec6-6727-42e4-82be-873d532ea659 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:19.807186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b6ec6-6727-42e4-82be-873d532ea659 None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:19.808442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598b6ec6-6727-42e4-82be-873d532ea659 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:19.810481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 100/206] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:19 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:19.816326 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b6ec6-6727-42e4-82be-873d532ea659 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:19.837534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:19.871742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cfa0a-9c2e-49ca-a733-b9ad617e238b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:19.872792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cfa0a-9c2e-49ca-a733-b9ad617e238b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:19.873183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cfa0a-9c2e-49ca-a733-b9ad617e238b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:19.873720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cfa0a-9c2e-49ca-a733-b9ad617e238b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:19.875098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cfa0a-9c2e-49ca-a733-b9ad617e238b None admin] RBAC: Adding query filter params (role.id=84bd13e4cee04c12bd6a0e549db79246, scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82, user.id=c3d7911f2f5f4f5386cd51a788182aa7) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:19.875098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cfa0a-9c2e-49ca-a733-b9ad617e238b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:19.876794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cfa0a-9c2e-49ca-a733-b9ad617e238b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:19.884247 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 107/207] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:19 2022] GET /identity/v3/role_assignments?user.id=c3d7911f2f5f4f5386cd51a788182aa7&scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82&role.id=84bd13e4cee04c12bd6a0e549db79246 => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:19.947437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b6ec6-6727-42e4-82be-873d532ea659 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:19.992815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ee1b63-96ed-4a7a-9988-2ff702e1383d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:19.993493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee1b63-96ed-4a7a-9988-2ff702e1383d None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:19.993785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee1b63-96ed-4a7a-9988-2ff702e1383d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:19.994320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ee1b63-96ed-4a7a-9988-2ff702e1383d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:19.995849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee1b63-96ed-4a7a-9988-2ff702e1383d None admin] RBAC: Adding query filter params (role.id=8427479f632a4da3949971f58b438c7f, scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2, user.id=38cd7f6cad2e43158de48fde6325e0f9) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:19.996285 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee1b63-96ed-4a7a-9988-2ff702e1383d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:19.997777 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ee1b63-96ed-4a7a-9988-2ff702e1383d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:20.011219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 101/208] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:19 2022] GET /identity/v3/role_assignments?user.id=38cd7f6cad2e43158de48fde6325e0f9&scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2&role.id=8427479f632a4da3949971f58b438c7f => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:20.811863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce772f64-4233-4c16-a32f-db51fd15169b None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:20.812434 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce772f64-4233-4c16-a32f-db51fd15169b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:20.812434 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce772f64-4233-4c16-a32f-db51fd15169b None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:20.813989 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 108/209] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:20.819520 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bd960-785e-4df0-8ab9-15d905b4c4af None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:20.819969 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bd960-785e-4df0-8ab9-15d905b4c4af None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:20.820148 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bd960-785e-4df0-8ab9-15d905b4c4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:20.837341 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-f98bd960-785e-4df0-8ab9-15d905b4c4af None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:20.838545 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f98bd960-785e-4df0-8ab9-15d905b4c4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:20.905946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbd349-9cc9-4199-9567-9b4fd2075209 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:20.906232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbd349-9cc9-4199-9567-9b4fd2075209 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:20.906511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbd349-9cc9-4199-9567-9b4fd2075209 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:20.907896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 109/210] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:37:20.913025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35b72c-9d44-4ab8-97b3-2ededeb3707f None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:20.913316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35b72c-9d44-4ab8-97b3-2ededeb3707f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:20.913594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35b72c-9d44-4ab8-97b3-2ededeb3707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:20.927595 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 102/211] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:20.936320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-8c35b72c-9d44-4ab8-97b3-2ededeb3707f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:20.937627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8c35b72c-9d44-4ab8-97b3-2ededeb3707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:20.951024 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98bd960-785e-4df0-8ab9-15d905b4c4af None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:20.965442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:21.019663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 110/212] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:21.044342 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35b72c-9d44-4ab8-97b3-2ededeb3707f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:21.067903 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98bd960-785e-4df0-8ab9-15d905b4c4af None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:21.103554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdffbb1-a728-4d70-8d8d-b4de8df5e4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:21.104381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdffbb1-a728-4d70-8d8d-b4de8df5e4bb None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:21.104694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdffbb1-a728-4d70-8d8d-b4de8df5e4bb None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:21.104966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdffbb1-a728-4d70-8d8d-b4de8df5e4bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:21.117844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdffbb1-a728-4d70-8d8d-b4de8df5e4bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-1cdffbb1-a728-4d70-8d8d-b4de8df5e4bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:21.124233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:21.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:21.125904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:21.125904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:21.125904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:21.125904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:21.125904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:21.125904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:21.125904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:21.125904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:21.125904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:21.125904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:21.125904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:21.125904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:21.125904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 103/213] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:21.132880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdffbb1-a728-4d70-8d8d-b4de8df5e4bb None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:21.141383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:21.171183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35b72c-9d44-4ab8-97b3-2ededeb3707f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:21.209080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf94e8-14e0-4b70-802e-6c7d82a36091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:21.210009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf94e8-14e0-4b70-802e-6c7d82a36091 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:21.210359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf94e8-14e0-4b70-802e-6c7d82a36091 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:21.210849 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf94e8-14e0-4b70-802e-6c7d82a36091 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:21.212336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf94e8-14e0-4b70-802e-6c7d82a36091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:21.214161 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf94e8-14e0-4b70-802e-6c7d82a36091 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:21.216572 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 111/214] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:37:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:21.222663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf94e8-14e0-4b70-802e-6c7d82a36091 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:21.228963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:21.269248 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdffbb1-a728-4d70-8d8d-b4de8df5e4bb None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:21.310833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f937fb-7699-4ba6-b1fa-78ff9919f6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:21.311866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f937fb-7699-4ba6-b1fa-78ff9919f6d2 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:21.312375 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f937fb-7699-4ba6-b1fa-78ff9919f6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:21.312875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f937fb-7699-4ba6-b1fa-78ff9919f6d2 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:21.313824 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f937fb-7699-4ba6-b1fa-78ff9919f6d2 None admin] RBAC: Adding query filter params (name=member) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:21.314344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f937fb-7699-4ba6-b1fa-78ff9919f6d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:21.317817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f937fb-7699-4ba6-b1fa-78ff9919f6d2 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:21.332409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 104/215] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:21.334070 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f937fb-7699-4ba6-b1fa-78ff9919f6d2 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:21.342384 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf94e8-14e0-4b70-802e-6c7d82a36091 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:21.379421 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a549c-21c9-4ab4-8114-beee58941646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:21.380435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a549c-21c9-4ab4-8114-beee58941646 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:21.380870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a549c-21c9-4ab4-8114-beee58941646 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:21.381326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a549c-21c9-4ab4-8114-beee58941646 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:21.382192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a549c-21c9-4ab4-8114-beee58941646 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:21.383877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a549c-21c9-4ab4-8114-beee58941646 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:21.401659 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:21.413962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 112/216] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:37:21 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:37:21.462732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f937fb-7699-4ba6-b1fa-78ff9919f6d2 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:21.500619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:21.507367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f014b-3132-4345-9f25-1be22b901401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:21.509617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f014b-3132-4345-9f25-1be22b901401 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:21.510117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f014b-3132-4345-9f25-1be22b901401 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:21.510631 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f014b-3132-4345-9f25-1be22b901401 None admin] PATH_INFO: `/v3/users/3bc6250c20504dfdb29bd6b765b71c96` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:21.527350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f014b-3132-4345-9f25-1be22b901401 None admin] RBAC: Authorizing `identity:get_user(user_id=3bc6250c20504dfdb29bd6b765b71c96)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:21.529434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f014b-3132-4345-9f25-1be22b901401 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:21.531768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 105/217] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:21 2022] GET /identity/v3/users/3bc6250c20504dfdb29bd6b765b71c96 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:21.544003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a549c-21c9-4ab4-8114-beee58941646 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:21.649804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a549c-21c9-4ab4-8114-beee58941646 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:21.687063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbff097-363e-4032-84af-2a5462c8bf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:21.687740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbff097-363e-4032-84af-2a5462c8bf6d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:21.688052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbff097-363e-4032-84af-2a5462c8bf6d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:21.688389 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbff097-363e-4032-84af-2a5462c8bf6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:21.757925 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:21.763034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbff097-363e-4032-84af-2a5462c8bf6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:21.764547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbff097-363e-4032-84af-2a5462c8bf6d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:21.874493 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 113/218] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:21.880259 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f014b-3132-4345-9f25-1be22b901401 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:21.985731 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a45eb3-fac4-4c13-b52b-4fd1f501e801 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:21.986018 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a45eb3-fac4-4c13-b52b-4fd1f501e801 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:21.986300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a45eb3-fac4-4c13-b52b-4fd1f501e801 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:21.987695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 114/219] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:37:21.993216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976c28c-0288-4e98-884f-379c840e55db None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:21.993509 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976c28c-0288-4e98-884f-379c840e55db None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:21.993791 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e976c28c-0288-4e98-884f-379c840e55db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:21.996777 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f014b-3132-4345-9f25-1be22b901401 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:22.024948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-e976c28c-0288-4e98-884f-379c840e55db None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:22.026158 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e976c28c-0288-4e98-884f-379c840e55db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:22.034217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d9afc-f9d7-42e3-8bee-59589e31ada1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:22.035487 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d9afc-f9d7-42e3-8bee-59589e31ada1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:22.036009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d9afc-f9d7-42e3-8bee-59589e31ada1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:22.036430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d9afc-f9d7-42e3-8bee-59589e31ada1 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:22.037820 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d9afc-f9d7-42e3-8bee-59589e31ada1 None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:22.039472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d9afc-f9d7-42e3-8bee-59589e31ada1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:22.046604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 106/220] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:21 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:22.048228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d9afc-f9d7-42e3-8bee-59589e31ada1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:22.137418 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:22.179862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 115/221] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:22.204726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d9afc-f9d7-42e3-8bee-59589e31ada1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:22.211134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976c28c-0288-4e98-884f-379c840e55db None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:22.244282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:22.253249 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a34a8d6-77c0-4fd7-96c6-e4a91c7d006a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:22.254143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34a8d6-77c0-4fd7-96c6-e4a91c7d006a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:22.254580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34a8d6-77c0-4fd7-96c6-e4a91c7d006a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:22.255040 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a34a8d6-77c0-4fd7-96c6-e4a91c7d006a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:22.256080 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34a8d6-77c0-4fd7-96c6-e4a91c7d006a None admin] RBAC: Adding query filter params (role.id=fbd0f348273f4791b1813d1df24df10e, scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2, user.id=3bc6250c20504dfdb29bd6b765b71c96) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:22.256544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34a8d6-77c0-4fd7-96c6-e4a91c7d006a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:22.258148 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a34a8d6-77c0-4fd7-96c6-e4a91c7d006a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:22.270149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 107/222] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:22 2022] GET /identity/v3/role_assignments?user.id=3bc6250c20504dfdb29bd6b765b71c96&scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2&role.id=fbd0f348273f4791b1813d1df24df10e => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:22.271487 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2c2ba-05ee-42bf-aa02-012922009bfe None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:22.271776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2c2ba-05ee-42bf-aa02-012922009bfe None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:22.272098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2c2ba-05ee-42bf-aa02-012922009bfe None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:22.274107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 108/223] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:22.281159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791008e-7f52-4e44-bcf0-3fb27215e4dc None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:22.281470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791008e-7f52-4e44-bcf0-3fb27215e4dc None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:22.281733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791008e-7f52-4e44-bcf0-3fb27215e4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:22.302181 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5791008e-7f52-4e44-bcf0-3fb27215e4dc None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:22.304494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5791008e-7f52-4e44-bcf0-3fb27215e4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:22.389676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e976c28c-0288-4e98-884f-379c840e55db None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:22.445935 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 109/224] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:22.456469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad825e3-cea0-4b31-8c67-5291c53b8989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:22.457293 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad825e3-cea0-4b31-8c67-5291c53b8989 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:22.457580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad825e3-cea0-4b31-8c67-5291c53b8989 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:22.457860 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad825e3-cea0-4b31-8c67-5291c53b8989 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:22.475442 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad825e3-cea0-4b31-8c67-5291c53b8989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:22.482329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791008e-7f52-4e44-bcf0-3fb27215e4dc None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-9ad825e3-cea0-4b31-8c67-5291c53b8989 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:22.484103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:22.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:22.485788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:22.485788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:22.485788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:22.485788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:22.485788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:22.485788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:22.485788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:22.485788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:22.485788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:22.485788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:22.485788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:22.485788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:22.485788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 116/225] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:37:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:22.491527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad825e3-cea0-4b31-8c67-5291c53b8989 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:22.549157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:22.605607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5791008e-7f52-4e44-bcf0-3fb27215e4dc None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:22.645592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c910f-65b7-4a29-94fa-d22989cdf1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:22.646415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c910f-65b7-4a29-94fa-d22989cdf1fe None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:22.646958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c910f-65b7-4a29-94fa-d22989cdf1fe None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:22.647237 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c910f-65b7-4a29-94fa-d22989cdf1fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:22.659429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c910f-65b7-4a29-94fa-d22989cdf1fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-167c910f-65b7-4a29-94fa-d22989cdf1fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:22.669641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:22.670547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:22.672160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:22.672160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:22.672160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:22.672160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:22.672160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:22.672160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:22.672160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:22.672160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:22.672160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:22.672160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:22.672160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:22.672160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:22.672160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 110/226] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:37:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:22.678635 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c910f-65b7-4a29-94fa-d22989cdf1fe None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:22.685174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:22.699086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad825e3-cea0-4b31-8c67-5291c53b8989 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:22.746282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8792f0-4930-429a-b7b6-d60220903fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:22.746965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8792f0-4930-429a-b7b6-d60220903fc7 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:22.747253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8792f0-4930-429a-b7b6-d60220903fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:22.747530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8792f0-4930-429a-b7b6-d60220903fc7 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:22.748322 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8792f0-4930-429a-b7b6-d60220903fc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:22.748631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8792f0-4930-429a-b7b6-d60220903fc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:22.750030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8792f0-4930-429a-b7b6-d60220903fc7 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:22.767999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 117/227] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:22.773152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8792f0-4930-429a-b7b6-d60220903fc7 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:22.852197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c910f-65b7-4a29-94fa-d22989cdf1fe None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:22.877531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:22.913031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d1e1e-d184-4753-a948-7688dbbad689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:22.913801 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d1e1e-d184-4753-a948-7688dbbad689 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:22.913987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d1e1e-d184-4753-a948-7688dbbad689 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:22.914274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d1e1e-d184-4753-a948-7688dbbad689 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:22.915045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d1e1e-d184-4753-a948-7688dbbad689 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:22.915356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d1e1e-d184-4753-a948-7688dbbad689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:22.918061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d1e1e-d184-4753-a948-7688dbbad689 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:22.934200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 111/228] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:37:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:22.938940 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d1e1e-d184-4753-a948-7688dbbad689 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:22.965156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8792f0-4930-429a-b7b6-d60220903fc7 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:23.039073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97137f-2643-4596-b266-e81717180819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:23.039986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97137f-2643-4596-b266-e81717180819 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:23.040275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97137f-2643-4596-b266-e81717180819 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:23.040555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df97137f-2643-4596-b266-e81717180819 None admin] PATH_INFO: `/v3/users/38cd7f6cad2e43158de48fde6325e0f9` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:23.043365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97137f-2643-4596-b266-e81717180819 None admin] RBAC: Authorizing `identity:get_user(user_id=38cd7f6cad2e43158de48fde6325e0f9)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:23.044916 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df97137f-2643-4596-b266-e81717180819 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:23.048230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 118/229] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:22 2022] GET /identity/v3/users/38cd7f6cad2e43158de48fde6325e0f9 => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:23.058416 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97137f-2643-4596-b266-e81717180819 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:23.098974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:23.128585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d1e1e-d184-4753-a948-7688dbbad689 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:23.171224 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700f068-f019-42dc-8299-01ee5699e4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:23.172149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700f068-f019-42dc-8299-01ee5699e4b0 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:23.172435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700f068-f019-42dc-8299-01ee5699e4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:23.172717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f700f068-f019-42dc-8299-01ee5699e4b0 None admin] PATH_INFO: `/v3/users/38cd7f6cad2e43158de48fde6325e0f9` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:23.174969 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f700f068-f019-42dc-8299-01ee5699e4b0 None admin] RBAC: Authorizing `identity:get_user(user_id=38cd7f6cad2e43158de48fde6325e0f9)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:23.177220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f700f068-f019-42dc-8299-01ee5699e4b0 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:23.181144 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 112/230] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:22 2022] GET /identity/v3/users/38cd7f6cad2e43158de48fde6325e0f9 => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:23.186106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700f068-f019-42dc-8299-01ee5699e4b0 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:23.254864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df97137f-2643-4596-b266-e81717180819 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:23.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:23.314347 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f700f068-f019-42dc-8299-01ee5699e4b0 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:23.317161 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e64a7-1d0b-4b0c-b3b7-f727b778408f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:23.318040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e64a7-1d0b-4b0c-b3b7-f727b778408f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:23.318320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e64a7-1d0b-4b0c-b3b7-f727b778408f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:23.318601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e64a7-1d0b-4b0c-b3b7-f727b778408f None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:23.324661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e64a7-1d0b-4b0c-b3b7-f727b778408f None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:23.326165 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e64a7-1d0b-4b0c-b3b7-f727b778408f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:23.327868 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 119/231] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:23 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:23.344968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e64a7-1d0b-4b0c-b3b7-f727b778408f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:23.356298 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456b37f-58db-43a8-8b32-77ea7bc333b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:23.357428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456b37f-58db-43a8-8b32-77ea7bc333b6 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:23.357929 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456b37f-58db-43a8-8b32-77ea7bc333b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:23.358356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7456b37f-58db-43a8-8b32-77ea7bc333b6 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:23.359719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456b37f-58db-43a8-8b32-77ea7bc333b6 None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:23.361389 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7456b37f-58db-43a8-8b32-77ea7bc333b6 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:23.363622 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 113/232] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:23 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:23.372864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456b37f-58db-43a8-8b32-77ea7bc333b6 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:23.473977 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7456b37f-58db-43a8-8b32-77ea7bc333b6 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:23.486473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:23.510095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d56bb-6ceb-4540-b068-00d9c463f458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:23.511774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d56bb-6ceb-4540-b068-00d9c463f458 None admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:23.512428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d56bb-6ceb-4540-b068-00d9c463f458 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:23.513030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d56bb-6ceb-4540-b068-00d9c463f458 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2/users/38cd7f6cad2e43158de48fde6325e0f9/roles/8427479f632a4da3949971f58b438c7f` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:23.521878 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d56bb-6ceb-4540-b068-00d9c463f458 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bebbd67fc624e7ea047e2d52573e5b2, user_id=38cd7f6cad2e43158de48fde6325e0f9, role_id=8427479f632a4da3949971f58b438c7f)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:23.523544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d56bb-6ceb-4540-b068-00d9c463f458 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:23.533297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:23.553070 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 114/233] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:37:23 2022] PUT /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2/users/38cd7f6cad2e43158de48fde6325e0f9/roles/8427479f632a4da3949971f58b438c7f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:37:23.557064 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e64a7-1d0b-4b0c-b3b7-f727b778408f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:23.608552 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10405f8e-2ab3-4f38-9ebc-b8fcd0571dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:23.609707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10405f8e-2ab3-4f38-9ebc-b8fcd0571dc6 None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:23.610191 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10405f8e-2ab3-4f38-9ebc-b8fcd0571dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:23.610716 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10405f8e-2ab3-4f38-9ebc-b8fcd0571dc6 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2/users/38cd7f6cad2e43158de48fde6325e0f9/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:23.614639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10405f8e-2ab3-4f38-9ebc-b8fcd0571dc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bebbd67fc624e7ea047e2d52573e5b2, user_id=38cd7f6cad2e43158de48fde6325e0f9, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:23.616340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10405f8e-2ab3-4f38-9ebc-b8fcd0571dc6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:23.643037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 120/234] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:37:23 2022] PUT /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2/users/38cd7f6cad2e43158de48fde6325e0f9/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:37:23.989440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbf9af-2fa3-4912-8551-d82b5a54595c None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:23.990314 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbf9af-2fa3-4912-8551-d82b5a54595c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:23.990744 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbf9af-2fa3-4912-8551-d82b5a54595c None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:23.992446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 115/235] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:23.998762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5153b-1ea7-4268-b229-14e57f02a138 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:23.999551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5153b-1ea7-4268-b229-14e57f02a138 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:24.000029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c5153b-1ea7-4268-b229-14e57f02a138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:24.025014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-84c5153b-1ea7-4268-b229-14e57f02a138 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:24.026479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-84c5153b-1ea7-4268-b229-14e57f02a138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:24.128473 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 121/236] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:24.164317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbf9af-2fa3-4912-8551-d82b5a54595c None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:24.220886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31755c52-a9cc-48e2-a9b9-299229512cf7 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:24.221406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31755c52-a9cc-48e2-a9b9-299229512cf7 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:24.221833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31755c52-a9cc-48e2-a9b9-299229512cf7 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:24.223392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 122/237] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:24.229153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e9340-c1af-495a-b4b5-628ee4b3dcb1 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:24.229443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e9340-c1af-495a-b4b5-628ee4b3dcb1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:24.230260 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5e9340-c1af-495a-b4b5-628ee4b3dcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:24.250828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-0c5e9340-c1af-495a-b4b5-628ee4b3dcb1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:24.252099 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0c5e9340-c1af-495a-b4b5-628ee4b3dcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:24.330478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbf9af-2fa3-4912-8551-d82b5a54595c None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:24.350750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 123/238] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:24.376144 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bc9eb-5b1f-4461-96a8-7040fb9627f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:24.377000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bc9eb-5b1f-4461-96a8-7040fb9627f2 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:24.377287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bc9eb-5b1f-4461-96a8-7040fb9627f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:24.377565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bc9eb-5b1f-4461-96a8-7040fb9627f2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:24.388854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bc9eb-5b1f-4461-96a8-7040fb9627f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:24.395897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e9340-c1af-495a-b4b5-628ee4b3dcb1 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-e73bc9eb-5b1f-4461-96a8-7040fb9627f2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:24.396980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:24.397879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:24.398665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:24.398665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:24.398665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:24.398665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:24.398665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:24.398665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:24.398665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:24.398665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:24.398665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:24.398665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:24.398665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:24.398665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:24.399270 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 116/239] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:24.405187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bc9eb-5b1f-4461-96a8-7040fb9627f2 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:24.420296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:24.478890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:24.582138 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5e9340-c1af-495a-b4b5-628ee4b3dcb1 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:24.585318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bc9eb-5b1f-4461-96a8-7040fb9627f2 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:24.620220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d8056b-9e7d-438d-be3d-d9e71e409829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:24.621619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8056b-9e7d-438d-be3d-d9e71e409829 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:24.621910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8056b-9e7d-438d-be3d-d9e71e409829 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:24.622184 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d8056b-9e7d-438d-be3d-d9e71e409829 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:24.623361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8056b-9e7d-438d-be3d-d9e71e409829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:24.624863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d8056b-9e7d-438d-be3d-d9e71e409829 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:24.625510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4e28b-02a8-47be-b3a0-2d90d4259b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:24.626398 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4e28b-02a8-47be-b3a0-2d90d4259b17 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:24.626827 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 124/240] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:37:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:24.627089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4e28b-02a8-47be-b3a0-2d90d4259b17 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:24.627561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4e28b-02a8-47be-b3a0-2d90d4259b17 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:24.628082 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebafe0e-bd70-4240-876d-75dbf3dc2ab6 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:24.628371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebafe0e-bd70-4240-876d-75dbf3dc2ab6 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:24.628650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebafe0e-bd70-4240-876d-75dbf3dc2ab6 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:24.629229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4e28b-02a8-47be-b3a0-2d90d4259b17 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:24.629741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4e28b-02a8-47be-b3a0-2d90d4259b17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:24.630110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 125/241] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:24.631691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4e28b-02a8-47be-b3a0-2d90d4259b17 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:24.635605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebafe0e-bd70-4240-876d-75dbf3dc2ab6 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:24.642726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 117/242] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:24.644222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc6ee0-e7d3-4b95-8bca-36fc350fb218 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:24.644703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc6ee0-e7d3-4b95-8bca-36fc350fb218 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:24.645147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc6ee0-e7d3-4b95-8bca-36fc350fb218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:24.663821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-d5cc6ee0-e7d3-4b95-8bca-36fc350fb218 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:24.665410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d5cc6ee0-e7d3-4b95-8bca-36fc350fb218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:24.739726 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:24.756036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 118/243] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:24.756036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc6ee0-e7d3-4b95-8bca-36fc350fb218 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:24.790836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:24.796872 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebafe0e-bd70-4240-876d-75dbf3dc2ab6 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:24.838054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de6b6a-2f80-448a-be28-66e037d2be66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:24.839508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6b6a-2f80-448a-be28-66e037d2be66 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:24.839996 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6b6a-2f80-448a-be28-66e037d2be66 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:24.840422 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de6b6a-2f80-448a-be28-66e037d2be66 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:24.841272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de6b6a-2f80-448a-be28-66e037d2be66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:24.842941 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de6b6a-2f80-448a-be28-66e037d2be66 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:24.868502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 126/244] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:37:24 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:37:24.870255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de6b6a-2f80-448a-be28-66e037d2be66 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:24.883529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc6ee0-e7d3-4b95-8bca-36fc350fb218 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:24.922402 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affe9bc-167f-4ba1-a100-1936f9199b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:24.924106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affe9bc-167f-4ba1-a100-1936f9199b6f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:24.924106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affe9bc-167f-4ba1-a100-1936f9199b6f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:24.924106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affe9bc-167f-4ba1-a100-1936f9199b6f None admin] PATH_INFO: `/v3/users/0c859050ce6740fc8ee0f081c5b4e9e7` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:24.939718 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affe9bc-167f-4ba1-a100-1936f9199b6f None admin] RBAC: Authorizing `identity:get_user(user_id=0c859050ce6740fc8ee0f081c5b4e9e7)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:24.941351 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affe9bc-167f-4ba1-a100-1936f9199b6f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:24.943290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 119/245] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:24 2022] GET /identity/v3/users/0c859050ce6740fc8ee0f081c5b4e9e7 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:24.948821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affe9bc-167f-4ba1-a100-1936f9199b6f None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:25.050413 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de6b6a-2f80-448a-be28-66e037d2be66 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:25.093125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:25.099303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457656be-54a5-4a7b-941d-8e093d319907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:25.101431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457656be-54a5-4a7b-941d-8e093d319907 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:25.101431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457656be-54a5-4a7b-941d-8e093d319907 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:25.101431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457656be-54a5-4a7b-941d-8e093d319907 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:25.123437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457656be-54a5-4a7b-941d-8e093d319907 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:25.125793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-457656be-54a5-4a7b-941d-8e093d319907 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:25.131392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:25.133240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:25.134030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:25.134030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:25.134030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:25.134030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:25.134030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:25.134030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:25.134030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:25.134030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:25.134030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:25.134030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:25.134030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:25.134030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:25.134030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 127/246] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:25.139668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457656be-54a5-4a7b-941d-8e093d319907 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:25.158791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affe9bc-167f-4ba1-a100-1936f9199b6f None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:25.201246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36850e4c-e937-4566-95cb-34a7463a95e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:25.202362 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36850e4c-e937-4566-95cb-34a7463a95e1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:25.202920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36850e4c-e937-4566-95cb-34a7463a95e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:25.203394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36850e4c-e937-4566-95cb-34a7463a95e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:25.264473 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457656be-54a5-4a7b-941d-8e093d319907 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:25.277899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36850e4c-e937-4566-95cb-34a7463a95e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:25.279712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36850e4c-e937-4566-95cb-34a7463a95e1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:25.302166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734cad5-83ba-46da-9b5c-348087fc5801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:25.302860 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734cad5-83ba-46da-9b5c-348087fc5801 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:25.303146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734cad5-83ba-46da-9b5c-348087fc5801 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:25.303429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734cad5-83ba-46da-9b5c-348087fc5801 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:25.304714 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734cad5-83ba-46da-9b5c-348087fc5801 None admin] RBAC: Adding query filter params (name=member) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:25.304714 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734cad5-83ba-46da-9b5c-348087fc5801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:25.306166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734cad5-83ba-46da-9b5c-348087fc5801 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:25.316985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 128/247] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:25.322591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734cad5-83ba-46da-9b5c-348087fc5801 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:25.391226 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:25.393175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:25.420782 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 120/248] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:25.426732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36850e4c-e937-4566-95cb-34a7463a95e1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:25.454456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734cad5-83ba-46da-9b5c-348087fc5801 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:25.493424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061de590-1f1a-4e7e-8b00-b774b9906431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:25.494885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061de590-1f1a-4e7e-8b00-b774b9906431 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:25.496014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061de590-1f1a-4e7e-8b00-b774b9906431 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:25.496492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061de590-1f1a-4e7e-8b00-b774b9906431 None admin] PATH_INFO: `/v3/users/3bc6250c20504dfdb29bd6b765b71c96` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:25.498047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061de590-1f1a-4e7e-8b00-b774b9906431 None admin] RBAC: Authorizing `identity:get_user(user_id=3bc6250c20504dfdb29bd6b765b71c96)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:25.499894 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061de590-1f1a-4e7e-8b00-b774b9906431 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:25.502273 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 129/249] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:25 2022] GET /identity/v3/users/3bc6250c20504dfdb29bd6b765b71c96 => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:25.508055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061de590-1f1a-4e7e-8b00-b774b9906431 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:25.546069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36850e4c-e937-4566-95cb-34a7463a95e1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:25.584463 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d7874-5fc1-4c16-91c1-b176ffb93e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:25.585445 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d7874-5fc1-4c16-91c1-b176ffb93e3c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:25.585805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d7874-5fc1-4c16-91c1-b176ffb93e3c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:25.586159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d7874-5fc1-4c16-91c1-b176ffb93e3c None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:25.588229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d7874-5fc1-4c16-91c1-b176ffb93e3c None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:25.589829 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d7874-5fc1-4c16-91c1-b176ffb93e3c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:25.591684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 121/250] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:25 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:25.597671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d7874-5fc1-4c16-91c1-b176ffb93e3c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:25.642148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061de590-1f1a-4e7e-8b00-b774b9906431 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:25.654658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:25.668341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:25.683820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482a657-236d-4a65-987b-09d902902f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:25.684831 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482a657-236d-4a65-987b-09d902902f77 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:25.685390 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482a657-236d-4a65-987b-09d902902f77 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:25.685671 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482a657-236d-4a65-987b-09d902902f77 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:25.687085 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482a657-236d-4a65-987b-09d902902f77 None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:25.689313 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482a657-236d-4a65-987b-09d902902f77 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:25.692687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 130/251] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:25 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:25.701340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482a657-236d-4a65-987b-09d902902f77 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:25.714741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d7874-5fc1-4c16-91c1-b176ffb93e3c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:25.754591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1737dc7d-f553-4ebf-8ad7-970bb9c39b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:25.755261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737dc7d-f553-4ebf-8ad7-970bb9c39b4d None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:25.755550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737dc7d-f553-4ebf-8ad7-970bb9c39b4d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:25.755831 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737dc7d-f553-4ebf-8ad7-970bb9c39b4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:25.756719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737dc7d-f553-4ebf-8ad7-970bb9c39b4d None admin] RBAC: Adding query filter params (role.id=84bd13e4cee04c12bd6a0e549db79246, scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2, user.id=0c859050ce6740fc8ee0f081c5b4e9e7) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:25.757040 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737dc7d-f553-4ebf-8ad7-970bb9c39b4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:25.758589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1737dc7d-f553-4ebf-8ad7-970bb9c39b4d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:25.766531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 122/252] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:25 2022] GET /identity/v3/role_assignments?user.id=0c859050ce6740fc8ee0f081c5b4e9e7&scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2&role.id=84bd13e4cee04c12bd6a0e549db79246 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:25.858998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482a657-236d-4a65-987b-09d902902f77 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:25.899509 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf88932-3d86-46bb-854f-b66747357eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:25.900452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf88932-3d86-46bb-854f-b66747357eeb None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:25.900737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf88932-3d86-46bb-854f-b66747357eeb None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:25.901020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf88932-3d86-46bb-854f-b66747357eeb None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2/users/3bc6250c20504dfdb29bd6b765b71c96/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:25.903290 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf88932-3d86-46bb-854f-b66747357eeb None admin] RBAC: Authorizing `identity:create_grant(project_id=1bebbd67fc624e7ea047e2d52573e5b2, user_id=3bc6250c20504dfdb29bd6b765b71c96, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:25.904836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf88932-3d86-46bb-854f-b66747357eeb None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:25.922616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 131/253] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:37:25 2022] PUT /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2/users/3bc6250c20504dfdb29bd6b765b71c96/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:37:26.367339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99678a-736a-4533-938a-ee8c1cd90f5c None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:26.367892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99678a-736a-4533-938a-ee8c1cd90f5c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:26.367892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb99678a-736a-4533-938a-ee8c1cd90f5c None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:26.369747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 123/254] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:26.375671 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89fa9c-c88b-4b5f-a464-672543640f93 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:26.376611 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89fa9c-c88b-4b5f-a464-672543640f93 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:26.377143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89fa9c-c88b-4b5f-a464-672543640f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:26.396216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-2a89fa9c-c88b-4b5f-a464-672543640f93 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:26.397780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-2a89fa9c-c88b-4b5f-a464-672543640f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:26.447691 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:26.483053 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 132/255] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:26.506931 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99678a-736a-4533-938a-ee8c1cd90f5c None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:26.650139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb99678a-736a-4533-938a-ee8c1cd90f5c None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:26.658230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e86c8-2091-434d-a81c-4aae75c566db None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:26.658624 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e86c8-2091-434d-a81c-4aae75c566db None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:26.658911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e86c8-2091-434d-a81c-4aae75c566db None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:26.660409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 133/256] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:26.666534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cf65c-b4de-4fab-a35f-23981f308154 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:26.666820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cf65c-b4de-4fab-a35f-23981f308154 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:26.667645 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cf65c-b4de-4fab-a35f-23981f308154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:26.672100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:26.696585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-fa3cf65c-b4de-4fab-a35f-23981f308154 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:26.697797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-fa3cf65c-b4de-4fab-a35f-23981f308154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:26.708415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca77abf-e544-4039-a163-4233354658ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:26.709301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca77abf-e544-4039-a163-4233354658ad None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:26.709585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca77abf-e544-4039-a163-4233354658ad None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:26.709860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca77abf-e544-4039-a163-4233354658ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:26.732464 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca77abf-e544-4039-a163-4233354658ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-bca77abf-e544-4039-a163-4233354658ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:26.748597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:26.749529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:26.750309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:26.750309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:26.750309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:26.750309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:26.750309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:26.750309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:26.750309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:26.750309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:26.750309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:26.750309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:26.750309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:26.750309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:26.750309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 124/257] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:37:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:26.765637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca77abf-e544-4039-a163-4233354658ad None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:26.860932 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 134/258] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 14 17:37:26.890588 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cf65c-b4de-4fab-a35f-23981f308154 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:26.983179 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca77abf-e544-4039-a163-4233354658ad None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:27.058543 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cf65c-b4de-4fab-a35f-23981f308154 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:27.073621 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:27.084172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920e576-8685-4310-a713-df669b31ee82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:27.085253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920e576-8685-4310-a713-df669b31ee82 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:27.085742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920e576-8685-4310-a713-df669b31ee82 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:27.086230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920e576-8685-4310-a713-df669b31ee82 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:27.087220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920e576-8685-4310-a713-df669b31ee82 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:27.087757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920e576-8685-4310-a713-df669b31ee82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:27.089984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920e576-8685-4310-a713-df669b31ee82 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:27.108213 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 125/259] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:37:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:27.128184 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920e576-8685-4310-a713-df669b31ee82 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:27.132443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a36ac9-4cbf-4201-a4c6-febbef944b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:27.133269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a36ac9-4cbf-4201-a4c6-febbef944b43 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:27.133560 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a36ac9-4cbf-4201-a4c6-febbef944b43 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:27.133838 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a36ac9-4cbf-4201-a4c6-febbef944b43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:27.169244 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a36ac9-4cbf-4201-a4c6-febbef944b43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-14a36ac9-4cbf-4201-a4c6-febbef944b43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:27.184271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:27.185246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:27.186546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:27.186546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:27.186546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:27.186546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:27.186546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:27.186546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:27.186546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:27.186546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:27.186546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:27.186546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:27.186546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:27.186546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:27.186546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 135/260] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:37:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:27.191952 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:27.206766 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a36ac9-4cbf-4201-a4c6-febbef944b43 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:27.336147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920e576-8685-4310-a713-df669b31ee82 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:27.400706 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2afba-9a56-468b-b961-2ae239281bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:27.401657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2afba-9a56-468b-b961-2ae239281bbf None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:27.401947 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2afba-9a56-468b-b961-2ae239281bbf None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:27.402226 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd2afba-9a56-468b-b961-2ae239281bbf None admin] PATH_INFO: `/v3/users/38cd7f6cad2e43158de48fde6325e0f9` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:27.403556 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2afba-9a56-468b-b961-2ae239281bbf None admin] RBAC: Authorizing `identity:get_user(user_id=38cd7f6cad2e43158de48fde6325e0f9)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:27.405066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd2afba-9a56-468b-b961-2ae239281bbf None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:27.407082 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 126/261] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:27 2022] GET /identity/v3/users/38cd7f6cad2e43158de48fde6325e0f9 => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:27.412882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2afba-9a56-468b-b961-2ae239281bbf None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:27.452811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a36ac9-4cbf-4201-a4c6-febbef944b43 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:27.521922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5290767-e1c4-4372-a61b-62bc48f6dbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:27.522998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5290767-e1c4-4372-a61b-62bc48f6dbfd None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:27.523483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5290767-e1c4-4372-a61b-62bc48f6dbfd None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:27.523993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5290767-e1c4-4372-a61b-62bc48f6dbfd None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:27.524958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5290767-e1c4-4372-a61b-62bc48f6dbfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:27.525481 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5290767-e1c4-4372-a61b-62bc48f6dbfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:27.527729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5290767-e1c4-4372-a61b-62bc48f6dbfd None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:27.546916 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 136/262] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:27.552683 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5290767-e1c4-4372-a61b-62bc48f6dbfd None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:27.606502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:27.627770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd2afba-9a56-468b-b961-2ae239281bbf None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:27.629693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:27.693851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f55f5-f6cf-4d41-b9c1-130cf238a829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:27.694534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f55f5-f6cf-4d41-b9c1-130cf238a829 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:27.694827 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f55f5-f6cf-4d41-b9c1-130cf238a829 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:27.695115 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f55f5-f6cf-4d41-b9c1-130cf238a829 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:27.762478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5290767-e1c4-4372-a61b-62bc48f6dbfd None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:27.819671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f55f5-f6cf-4d41-b9c1-130cf238a829 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:27.821172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f55f5-f6cf-4d41-b9c1-130cf238a829 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:27.822491 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fde6f1-7d35-4fa1-99a4-9f6675946e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:27.824157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fde6f1-7d35-4fa1-99a4-9f6675946e71 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:27.824547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fde6f1-7d35-4fa1-99a4-9f6675946e71 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:27.824925 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fde6f1-7d35-4fa1-99a4-9f6675946e71 None admin] PATH_INFO: `/v3/users/38cd7f6cad2e43158de48fde6325e0f9` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:27.831951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fde6f1-7d35-4fa1-99a4-9f6675946e71 None admin] RBAC: Authorizing `identity:get_user(user_id=38cd7f6cad2e43158de48fde6325e0f9)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:27.831951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fde6f1-7d35-4fa1-99a4-9f6675946e71 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:27.831951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 137/263] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:27 2022] GET /identity/v3/users/38cd7f6cad2e43158de48fde6325e0f9 => generated 283 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:27.844038 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fde6f1-7d35-4fa1-99a4-9f6675946e71 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:28.054119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:28.061236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fde6f1-7d35-4fa1-99a4-9f6675946e71 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:28.080558 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 127/264] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 17:37:28.087051 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f55f5-f6cf-4d41-b9c1-130cf238a829 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:28.095959 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:28.156196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ef035-9729-4567-a79b-e50f2fa3933b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:28.156883 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef035-9729-4567-a79b-e50f2fa3933b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:28.157175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef035-9729-4567-a79b-e50f2fa3933b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:28.157457 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ef035-9729-4567-a79b-e50f2fa3933b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:28.264702 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f55f5-f6cf-4d41-b9c1-130cf238a829 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:28.308474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ef035-9729-4567-a79b-e50f2fa3933b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:28.310212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ef035-9729-4567-a79b-e50f2fa3933b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:28.333391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374c510-2221-42d2-b95b-3567baf6fc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:28.335412 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374c510-2221-42d2-b95b-3567baf6fc8c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:28.335971 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374c510-2221-42d2-b95b-3567baf6fc8c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:28.336475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374c510-2221-42d2-b95b-3567baf6fc8c None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:28.338186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374c510-2221-42d2-b95b-3567baf6fc8c None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:28.340577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374c510-2221-42d2-b95b-3567baf6fc8c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:28.350437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 128/265] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:28 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:28.355485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374c510-2221-42d2-b95b-3567baf6fc8c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:28.467102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:28.511579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374c510-2221-42d2-b95b-3567baf6fc8c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:28.521513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:28.536149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 138/266] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 693 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 17:37:28.537732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574d2ec-4446-41ec-b778-2bb66388eb9e None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:28.538026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574d2ec-4446-41ec-b778-2bb66388eb9e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:28.538306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574d2ec-4446-41ec-b778-2bb66388eb9e None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:28.540026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 139/267] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:28.540835 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574d2ec-4446-41ec-b778-2bb66388eb9e None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:28.564340 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c288fa-d336-4bc4-9754-1d2441be75fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:28.565266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c288fa-d336-4bc4-9754-1d2441be75fe None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:28.565722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c288fa-d336-4bc4-9754-1d2441be75fe None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:28.566690 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c288fa-d336-4bc4-9754-1d2441be75fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:28.567775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c288fa-d336-4bc4-9754-1d2441be75fe None admin] RBAC: Adding query filter params (role.id=8427479f632a4da3949971f58b438c7f, scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2, user.id=38cd7f6cad2e43158de48fde6325e0f9) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:28.568343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c288fa-d336-4bc4-9754-1d2441be75fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:28.569979 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c288fa-d336-4bc4-9754-1d2441be75fe None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:28.577472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 129/268] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:28 2022] GET /identity/v3/role_assignments?user.id=38cd7f6cad2e43158de48fde6325e0f9&scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2&role.id=8427479f632a4da3949971f58b438c7f => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:28.579606 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724a83d-5193-4f58-bec6-c67cf9a8e8b2 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:28.580114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724a83d-5193-4f58-bec6-c67cf9a8e8b2 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:28.580541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724a83d-5193-4f58-bec6-c67cf9a8e8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:28.600640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-1724a83d-5193-4f58-bec6-c67cf9a8e8b2 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:28.602210 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1724a83d-5193-4f58-bec6-c67cf9a8e8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:28.686369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 130/269] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:28.687996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edacda-1816-4074-8ed6-ddaf09f6a81c None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:28.688437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edacda-1816-4074-8ed6-ddaf09f6a81c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:28.688863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edacda-1816-4074-8ed6-ddaf09f6a81c None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:28.691311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 131/270] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:28.698239 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a1b9a-5d0e-4f0e-b726-fc57ffb165b0 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:28.699067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a1b9a-5d0e-4f0e-b726-fc57ffb165b0 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:28.699500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a1b9a-5d0e-4f0e-b726-fc57ffb165b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:28.733171 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-d58a1b9a-5d0e-4f0e-b726-fc57ffb165b0 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:28.734394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d58a1b9a-5d0e-4f0e-b726-fc57ffb165b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:28.792013 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574d2ec-4446-41ec-b778-2bb66388eb9e None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:28.836514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e235fc-2166-406c-b81a-86c94d14eee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:28.837616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e235fc-2166-406c-b81a-86c94d14eee1 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:28.838061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e235fc-2166-406c-b81a-86c94d14eee1 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:28.838497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e235fc-2166-406c-b81a-86c94d14eee1 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:28.839907 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e235fc-2166-406c-b81a-86c94d14eee1 None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:28.842129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e235fc-2166-406c-b81a-86c94d14eee1 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:28.844370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 140/271] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:28 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:28.847981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e235fc-2166-406c-b81a-86c94d14eee1 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:28.851274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:28.861707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 132/272] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:28.863804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e9d56-57a1-4df9-b9f9-c599591e8492 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:28.864119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e9d56-57a1-4df9-b9f9-c599591e8492 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:28.864400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e9d56-57a1-4df9-b9f9-c599591e8492 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:28.865757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 133/273] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:28.866473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e9d56-57a1-4df9-b9f9-c599591e8492 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:28.931526 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:28.985080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e235fc-2166-406c-b81a-86c94d14eee1 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:28.991810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e9d56-57a1-4df9-b9f9-c599591e8492 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:29.026033 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7ff35-108c-4b9d-b407-693fa3fde0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:29.026906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7ff35-108c-4b9d-b407-693fa3fde0f1 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:29.027191 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7ff35-108c-4b9d-b407-693fa3fde0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:29.027534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f7ff35-108c-4b9d-b407-693fa3fde0f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:29.029017 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de704156-72fc-42a7-9382-258d96432c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:29.029679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de704156-72fc-42a7-9382-258d96432c1b None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:29.029966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de704156-72fc-42a7-9382-258d96432c1b None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:29.030248 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de704156-72fc-42a7-9382-258d96432c1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:29.031095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de704156-72fc-42a7-9382-258d96432c1b None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2, user.id=38cd7f6cad2e43158de48fde6325e0f9) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:29.031410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de704156-72fc-42a7-9382-258d96432c1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:29.032905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de704156-72fc-42a7-9382-258d96432c1b None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:29.039272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 134/274] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:28 2022] GET /identity/v3/role_assignments?user.id=38cd7f6cad2e43158de48fde6325e0f9&scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:29.040455 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f7ff35-108c-4b9d-b407-693fa3fde0f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:29.045495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e56d76-e6b6-44bb-a7d1-e317a3de58b1 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:29.045863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e56d76-e6b6-44bb-a7d1-e317a3de58b1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:29.046212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e56d76-e6b6-44bb-a7d1-e317a3de58b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-d6f7ff35-108c-4b9d-b407-693fa3fde0f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:29.050040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:29.050988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:29.051793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:29.051793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:29.051793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:29.051793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:29.051793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:29.051793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:29.051793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:29.051793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:29.051793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:29.051793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:29.051793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:29.051793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:29.060069 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 141/275] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:29.060069 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7ff35-108c-4b9d-b407-693fa3fde0f1 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:29.073695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-22e56d76-e6b6-44bb-a7d1-e317a3de58b1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:29.075003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-22e56d76-e6b6-44bb-a7d1-e317a3de58b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:29.163450 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:29.183672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 135/276] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:29.184664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e56d76-e6b6-44bb-a7d1-e317a3de58b1 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:29.210328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f7ff35-108c-4b9d-b407-693fa3fde0f1 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:29.242632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:29.253523 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f510551b-db8c-44cc-b2f6-40b42afec2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:29.254355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510551b-db8c-44cc-b2f6-40b42afec2dd None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:29.254641 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510551b-db8c-44cc-b2f6-40b42afec2dd None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:29.254922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f510551b-db8c-44cc-b2f6-40b42afec2dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:29.271295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f510551b-db8c-44cc-b2f6-40b42afec2dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-f510551b-db8c-44cc-b2f6-40b42afec2dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:29.279892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:29.280873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:29.281665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:29.281665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:29.281665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:29.281665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:29.281665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:29.281665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:29.281665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:29.281665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:29.281665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:29.281665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:29.281665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:29.281665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:29.281665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 142/277] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:29.282647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b81dff-c233-43e8-9cfa-6d3562e76c42 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:29.282829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b81dff-c233-43e8-9cfa-6d3562e76c42 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:29.283046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b81dff-c233-43e8-9cfa-6d3562e76c42 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:29.284501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 143/278] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:29.289005 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b81dff-c233-43e8-9cfa-6d3562e76c42 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:29.307647 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e56d76-e6b6-44bb-a7d1-e317a3de58b1 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:29.342651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622105ae-bafb-474f-82cf-6b4c29525a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:29.343329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622105ae-bafb-474f-82cf-6b4c29525a1f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:29.343620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622105ae-bafb-474f-82cf-6b4c29525a1f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:29.343893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622105ae-bafb-474f-82cf-6b4c29525a1f None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:29.344693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622105ae-bafb-474f-82cf-6b4c29525a1f None admin] RBAC: Adding query filter params (name=member) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:29.345004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622105ae-bafb-474f-82cf-6b4c29525a1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:29.346410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622105ae-bafb-474f-82cf-6b4c29525a1f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:29.355883 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 136/279] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:29.357066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622105ae-bafb-474f-82cf-6b4c29525a1f None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:29.393353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b81dff-c233-43e8-9cfa-6d3562e76c42 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:29.406944 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:29.428565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2520e3-5ad4-4c2d-82f3-ec9a0885d799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:29.429373 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2520e3-5ad4-4c2d-82f3-ec9a0885d799 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:29.429663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2520e3-5ad4-4c2d-82f3-ec9a0885d799 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:29.429944 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2520e3-5ad4-4c2d-82f3-ec9a0885d799 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:29.431124 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2520e3-5ad4-4c2d-82f3-ec9a0885d799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:29.432633 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2520e3-5ad4-4c2d-82f3-ec9a0885d799 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:29.434272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 144/280] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:37:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:29.435539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6db3a4-a88f-4765-8063-6f224775c927 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:29.435830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6db3a4-a88f-4765-8063-6f224775c927 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:29.436136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6db3a4-a88f-4765-8063-6f224775c927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:29.453101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-de6db3a4-a88f-4765-8063-6f224775c927 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:29.454308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-de6db3a4-a88f-4765-8063-6f224775c927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:29.461449 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622105ae-bafb-474f-82cf-6b4c29525a1f None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:29.496677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fb643-158c-4115-b9e0-fbbbb90cbfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:29.497369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fb643-158c-4115-b9e0-fbbbb90cbfb0 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:29.497622 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fb643-158c-4115-b9e0-fbbbb90cbfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:29.497901 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924fb643-158c-4115-b9e0-fbbbb90cbfb0 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:29.498661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924fb643-158c-4115-b9e0-fbbbb90cbfb0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:29.498969 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924fb643-158c-4115-b9e0-fbbbb90cbfb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:29.500386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924fb643-158c-4115-b9e0-fbbbb90cbfb0 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:29.505298 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:29.509450 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 137/281] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:29.510377 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fb643-158c-4115-b9e0-fbbbb90cbfb0 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:29.524076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 145/282] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:29.525025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6db3a4-a88f-4765-8063-6f224775c927 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:29.622425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924fb643-158c-4115-b9e0-fbbbb90cbfb0 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:29.632346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6db3a4-a88f-4765-8063-6f224775c927 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:29.635278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:29.656375 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ad4de-a49d-47a4-ade5-3a6202e52bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:29.657170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ad4de-a49d-47a4-ade5-3a6202e52bc6 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:29.657449 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ad4de-a49d-47a4-ade5-3a6202e52bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:29.657725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ad4de-a49d-47a4-ade5-3a6202e52bc6 None admin] PATH_INFO: `/v3/users/d81458a88938477092806576f4b080ff` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:29.668056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b0864-7ae4-4663-b7bb-4c6e5f3a4f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:29.668745 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b0864-7ae4-4663-b7bb-4c6e5f3a4f0a None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:29.669026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b0864-7ae4-4663-b7bb-4c6e5f3a4f0a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:29.669295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b0864-7ae4-4663-b7bb-4c6e5f3a4f0a None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:29.670004 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b0864-7ae4-4663-b7bb-4c6e5f3a4f0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:29.671466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b0864-7ae4-4663-b7bb-4c6e5f3a4f0a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:29.671726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ad4de-a49d-47a4-ade5-3a6202e52bc6 None admin] RBAC: Authorizing `identity:get_user(user_id=d81458a88938477092806576f4b080ff)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:29.673225 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ad4de-a49d-47a4-ade5-3a6202e52bc6 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:29.674936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 138/283] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:29 2022] GET /identity/v3/users/d81458a88938477092806576f4b080ff => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:29.675701 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ad4de-a49d-47a4-ade5-3a6202e52bc6 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:29.693017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 146/284] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:37:29 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:37:29.693868 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b0864-7ae4-4663-b7bb-4c6e5f3a4f0a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:29.742753 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:29.799664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ad4de-a49d-47a4-ade5-3a6202e52bc6 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:29.818269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b0864-7ae4-4663-b7bb-4c6e5f3a4f0a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:29.838730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d5c07-c29c-4afd-b5f8-fca322739fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:29.840220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d5c07-c29c-4afd-b5f8-fca322739fda None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:29.840503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d5c07-c29c-4afd-b5f8-fca322739fda None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:29.840792 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7d5c07-c29c-4afd-b5f8-fca322739fda None admin] PATH_INFO: `/v3/users/0c859050ce6740fc8ee0f081c5b4e9e7` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:29.842099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d5c07-c29c-4afd-b5f8-fca322739fda None admin] RBAC: Authorizing `identity:get_user(user_id=0c859050ce6740fc8ee0f081c5b4e9e7)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:29.843610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7d5c07-c29c-4afd-b5f8-fca322739fda None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:29.845538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 139/285] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:29 2022] GET /identity/v3/users/0c859050ce6740fc8ee0f081c5b4e9e7 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:29.846179 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d5c07-c29c-4afd-b5f8-fca322739fda None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:29.864550 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7ce04-3747-4312-b10e-37a28aefacce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:29.865365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7ce04-3747-4312-b10e-37a28aefacce None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:29.866412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7ce04-3747-4312-b10e-37a28aefacce None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:29.866412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7ce04-3747-4312-b10e-37a28aefacce None admin] PATH_INFO: `/v3/roles/member` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:29.879767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7ce04-3747-4312-b10e-37a28aefacce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-c1a7ce04-3747-4312-b10e-37a28aefacce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:29.888344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:29.889270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:29.890202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:29.890202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:29.890202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:29.890202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:29.890202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:29.890202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:29.890202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:29.890202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:29.890202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:29.890202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:29.890202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:29.890202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:29.890202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 147/286] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:29.890875 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7ce04-3747-4312-b10e-37a28aefacce None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:29.899401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:29.983010 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7d5c07-c29c-4afd-b5f8-fca322739fda None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:30.022904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:30.027757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7ce04-3747-4312-b10e-37a28aefacce None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:30.031232 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc963f-9e54-45d0-b370-6f73959fca97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:30.031906 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc963f-9e54-45d0-b370-6f73959fca97 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:30.032247 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc963f-9e54-45d0-b370-6f73959fca97 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:30.032521 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc963f-9e54-45d0-b370-6f73959fca97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:30.033358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc963f-9e54-45d0-b370-6f73959fca97 None admin] RBAC: Adding query filter params (role.id=fbd0f348273f4791b1813d1df24df10e, user.id=d81458a88938477092806576f4b080ff, scope.system=all) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:30.033667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc963f-9e54-45d0-b370-6f73959fca97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:30.035129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc963f-9e54-45d0-b370-6f73959fca97 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:30.042098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 140/287] 10.222.0.64 () {56 vars in 1308 bytes} [Thu Jul 14 17:37:29 2022] GET /identity/v3/role_assignments?user.id=d81458a88938477092806576f4b080ff&scope.system=all&role.id=fbd0f348273f4791b1813d1df24df10e => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:30.043474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc963f-9e54-45d0-b370-6f73959fca97 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:30.075097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e420a0-16ed-4795-8367-d81f557c694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:30.076439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e420a0-16ed-4795-8367-d81f557c694f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:30.076995 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e420a0-16ed-4795-8367-d81f557c694f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:30.077483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e420a0-16ed-4795-8367-d81f557c694f None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:30.079034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e420a0-16ed-4795-8367-d81f557c694f None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:30.080802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e420a0-16ed-4795-8367-d81f557c694f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:30.082858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 148/288] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:29 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:30.097405 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e420a0-16ed-4795-8367-d81f557c694f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:30.193445 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc963f-9e54-45d0-b370-6f73959fca97 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:30.200337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:30.264210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e420a0-16ed-4795-8367-d81f557c694f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:30.281976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c00ad0-82a6-47a8-820d-6dad651bfee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:30.283030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c00ad0-82a6-47a8-820d-6dad651bfee4 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:30.283532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c00ad0-82a6-47a8-820d-6dad651bfee4 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:30.283872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c00ad0-82a6-47a8-820d-6dad651bfee4 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:30.284749 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c00ad0-82a6-47a8-820d-6dad651bfee4 None admin] RBAC: Adding query filter params (name=member) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:30.285130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c00ad0-82a6-47a8-820d-6dad651bfee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:30.286697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c00ad0-82a6-47a8-820d-6dad651bfee4 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:30.305891 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 141/289] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:30.314087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c00ad0-82a6-47a8-820d-6dad651bfee4 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:30.322399 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a5e30-dd45-456f-a8c9-0f2897d88acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:30.324235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a5e30-dd45-456f-a8c9-0f2897d88acf None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:30.324724 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a5e30-dd45-456f-a8c9-0f2897d88acf None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:30.325198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0a5e30-dd45-456f-a8c9-0f2897d88acf None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2/users/0c859050ce6740fc8ee0f081c5b4e9e7/roles/84bd13e4cee04c12bd6a0e549db79246` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:30.334006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a5e30-dd45-456f-a8c9-0f2897d88acf None admin] RBAC: Authorizing `identity:create_grant(project_id=1bebbd67fc624e7ea047e2d52573e5b2, user_id=0c859050ce6740fc8ee0f081c5b4e9e7, role_id=84bd13e4cee04c12bd6a0e549db79246)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:30.335739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0a5e30-dd45-456f-a8c9-0f2897d88acf None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:30.356538 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 149/290] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:37:30 2022] PUT /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2/users/0c859050ce6740fc8ee0f081c5b4e9e7/roles/84bd13e4cee04c12bd6a0e549db79246 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:37:30.497504 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c00ad0-82a6-47a8-820d-6dad651bfee4 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:30.554670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03a7a4-4fcf-4500-a688-08114ab93471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:30.555490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03a7a4-4fcf-4500-a688-08114ab93471 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:30.555775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03a7a4-4fcf-4500-a688-08114ab93471 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:30.556081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03a7a4-4fcf-4500-a688-08114ab93471 None admin] PATH_INFO: `/v3/users/3bc6250c20504dfdb29bd6b765b71c96` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:30.557362 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03a7a4-4fcf-4500-a688-08114ab93471 None admin] RBAC: Authorizing `identity:get_user(user_id=3bc6250c20504dfdb29bd6b765b71c96)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:30.558876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03a7a4-4fcf-4500-a688-08114ab93471 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:30.560896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 142/291] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:30 2022] GET /identity/v3/users/3bc6250c20504dfdb29bd6b765b71c96 => generated 297 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:30.566649 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a5e30-dd45-456f-a8c9-0f2897d88acf None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:30.607339 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:30.689173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0a5e30-dd45-456f-a8c9-0f2897d88acf None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:30.726013 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6419a75-0d21-4567-9530-6bec4a221291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:30.727053 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6419a75-0d21-4567-9530-6bec4a221291 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:30.727534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6419a75-0d21-4567-9530-6bec4a221291 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:30.728038 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6419a75-0d21-4567-9530-6bec4a221291 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:30.801592 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6419a75-0d21-4567-9530-6bec4a221291 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:30.803452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6419a75-0d21-4567-9530-6bec4a221291 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:30.832618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:30.909571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 150/292] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:30.917419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03a7a4-4fcf-4500-a688-08114ab93471 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:30.982031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:31.088939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03a7a4-4fcf-4500-a688-08114ab93471 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:31.142420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0f341-aba7-4aab-a5fd-9057e6283fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:31.143331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f341-aba7-4aab-a5fd-9057e6283fa9 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:31.143594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f341-aba7-4aab-a5fd-9057e6283fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:31.143877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0f341-aba7-4aab-a5fd-9057e6283fa9 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:31.148796 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0f341-aba7-4aab-a5fd-9057e6283fa9 None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:31.150292 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0f341-aba7-4aab-a5fd-9057e6283fa9 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:31.152770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 143/293] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:30 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:31.159459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6419a75-0d21-4567-9530-6bec4a221291 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:31.312060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6419a75-0d21-4567-9530-6bec4a221291 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:31.351183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:31.363579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57a979-6a18-4f5a-8709-4b454cbe0cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:31.364280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57a979-6a18-4f5a-8709-4b454cbe0cac None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:31.364567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57a979-6a18-4f5a-8709-4b454cbe0cac None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:31.364848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57a979-6a18-4f5a-8709-4b454cbe0cac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:31.365705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57a979-6a18-4f5a-8709-4b454cbe0cac None admin] RBAC: Adding query filter params (role.id=fbd0f348273f4791b1813d1df24df10e, scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2, user.id=3bc6250c20504dfdb29bd6b765b71c96) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:31.366022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57a979-6a18-4f5a-8709-4b454cbe0cac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:31.367485 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57a979-6a18-4f5a-8709-4b454cbe0cac None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:31.375134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 151/294] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:31 2022] GET /identity/v3/role_assignments?user.id=3bc6250c20504dfdb29bd6b765b71c96&scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2&role.id=fbd0f348273f4791b1813d1df24df10e => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:32.169158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79c98e-d306-4c56-9c6d-88999814f130 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:32.169692 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79c98e-d306-4c56-9c6d-88999814f130 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:32.169692 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79c98e-d306-4c56-9c6d-88999814f130 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:32.171091 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 144/295] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:32.176751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba8614-f5f3-4132-88e1-24d8e8b3de3f None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:32.177204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba8614-f5f3-4132-88e1-24d8e8b3de3f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:32.177403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba8614-f5f3-4132-88e1-24d8e8b3de3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:32.194946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b2ba8614-f5f3-4132-88e1-24d8e8b3de3f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:32.196174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b2ba8614-f5f3-4132-88e1-24d8e8b3de3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:32.270396 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 152/296] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:32.294705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79c98e-d306-4c56-9c6d-88999814f130 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:32.375192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a1abc-686a-4df8-bea9-7083ac86c45d None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:32.375790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a1abc-686a-4df8-bea9-7083ac86c45d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:32.376286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93a1abc-686a-4df8-bea9-7083ac86c45d None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:32.378086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 153/297] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:32.383535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad6c8a-3368-43ac-8138-76caea64dd93 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:32.384115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad6c8a-3368-43ac-8138-76caea64dd93 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:32.384628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad6c8a-3368-43ac-8138-76caea64dd93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:32.402913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-95ad6c8a-3368-43ac-8138-76caea64dd93 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:32.404472 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-95ad6c8a-3368-43ac-8138-76caea64dd93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:32.406160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:32.410251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79c98e-d306-4c56-9c6d-88999814f130 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:32.446456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507bb30-9f2d-4ba1-b596-5a6d9b72ea7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:32.447711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507bb30-9f2d-4ba1-b596-5a6d9b72ea7f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:32.447711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507bb30-9f2d-4ba1-b596-5a6d9b72ea7f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:32.447898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6507bb30-9f2d-4ba1-b596-5a6d9b72ea7f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:32.459287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6507bb30-9f2d-4ba1-b596-5a6d9b72ea7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-6507bb30-9f2d-4ba1-b596-5a6d9b72ea7f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:32.467678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:32.468607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:32.469400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:32.469400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:32.469400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:32.469400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:32.469400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:32.469400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:32.469400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:32.469400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:32.469400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:32.469400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:32.469400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:32.469400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:32.469400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 145/298] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:32.475129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507bb30-9f2d-4ba1-b596-5a6d9b72ea7f None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:32.486349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 154/299] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:32.510294 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad6c8a-3368-43ac-8138-76caea64dd93 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:32.565373 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:32.579845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6507bb30-9f2d-4ba1-b596-5a6d9b72ea7f None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:32.614957 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703698b-8c27-49be-bab2-35f5ae81daf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:32.615639 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703698b-8c27-49be-bab2-35f5ae81daf5 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:32.615946 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703698b-8c27-49be-bab2-35f5ae81daf5 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:32.616235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703698b-8c27-49be-bab2-35f5ae81daf5 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:32.616999 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703698b-8c27-49be-bab2-35f5ae81daf5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:32.617094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ad6c8a-3368-43ac-8138-76caea64dd93 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:32.617309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703698b-8c27-49be-bab2-35f5ae81daf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:32.618692 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703698b-8c27-49be-bab2-35f5ae81daf5 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:32.632329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 146/300] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:32.634420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd7d9a-a9a4-470b-96da-62d877535727 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:32.634806 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd7d9a-a9a4-470b-96da-62d877535727 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:32.635165 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bd7d9a-a9a4-470b-96da-62d877535727 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:32.637016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 147/301] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:32.637696 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd7d9a-a9a4-470b-96da-62d877535727 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:32.661495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c743a6-004c-4f50-b8ba-4af69c61bff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:32.662298 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c743a6-004c-4f50-b8ba-4af69c61bff5 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:32.662584 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c743a6-004c-4f50-b8ba-4af69c61bff5 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:32.662860 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c743a6-004c-4f50-b8ba-4af69c61bff5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:32.663615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:32.664057 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c743a6-004c-4f50-b8ba-4af69c61bff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:32.665557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c743a6-004c-4f50-b8ba-4af69c61bff5 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:32.667397 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 155/302] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:37:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:32.668427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0417d6-1ad0-4b58-91a8-abcfc895f1e2 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:32.668715 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0417d6-1ad0-4b58-91a8-abcfc895f1e2 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:32.668993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0417d6-1ad0-4b58-91a8-abcfc895f1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:32.686385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-1e0417d6-1ad0-4b58-91a8-abcfc895f1e2 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:32.687277 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1e0417d6-1ad0-4b58-91a8-abcfc895f1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:32.752608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bd7d9a-a9a4-470b-96da-62d877535727 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:32.762482 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 156/303] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:32.763587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0417d6-1ad0-4b58-91a8-abcfc895f1e2 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:32.799040 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b2c90-a76d-44b6-9463-87cdfe3b6c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:32.800280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b2c90-a76d-44b6-9463-87cdfe3b6c5f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:32.800567 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b2c90-a76d-44b6-9463-87cdfe3b6c5f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:32.800844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78b2c90-a76d-44b6-9463-87cdfe3b6c5f None admin] PATH_INFO: `/v3/users/5d81d732fa4148edacdb6aaaefa81cdc` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:32.816168 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b2c90-a76d-44b6-9463-87cdfe3b6c5f None admin] RBAC: Authorizing `identity:get_user(user_id=5d81d732fa4148edacdb6aaaefa81cdc)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:32.817633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78b2c90-a76d-44b6-9463-87cdfe3b6c5f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:32.819348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 148/304] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:32 2022] GET /identity/v3/users/5d81d732fa4148edacdb6aaaefa81cdc => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:32.820147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b2c90-a76d-44b6-9463-87cdfe3b6c5f None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:32.830813 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:32.886519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0417d6-1ad0-4b58-91a8-abcfc895f1e2 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:32.901302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:32.922233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78b2c90-a76d-44b6-9463-87cdfe3b6c5f None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:32.926132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efaf28-4f3e-47d9-a159-a80b35e5b3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:32.926973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efaf28-4f3e-47d9-a159-a80b35e5b3f7 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:32.927425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efaf28-4f3e-47d9-a159-a80b35e5b3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:32.927840 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38efaf28-4f3e-47d9-a159-a80b35e5b3f7 None admin] PATH_INFO: `/v3/groups` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:32.928687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efaf28-4f3e-47d9-a159-a80b35e5b3f7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:32.930395 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38efaf28-4f3e-47d9-a159-a80b35e5b3f7 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:32.955958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 157/305] 10.222.0.64 () {60 vars in 1134 bytes} [Thu Jul 14 17:37:32 2022] POST /identity/v3/groups => generated 229 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:37:32.955958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efaf28-4f3e-47d9-a159-a80b35e5b3f7 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:32.959804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0770947b-2787-489c-9869-216f4be17e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:32.960661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770947b-2787-489c-9869-216f4be17e58 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:32.960947 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770947b-2787-489c-9869-216f4be17e58 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:32.961221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0770947b-2787-489c-9869-216f4be17e58 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:32.974529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0770947b-2787-489c-9869-216f4be17e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-0770947b-2787-489c-9869-216f4be17e58 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:32.983158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:32.984350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:32.985170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:32.985170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:32.985170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:32.985170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:32.985170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:32.985170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:32.985170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:32.985170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:32.985170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:32.985170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:32.985170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:32.985170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:32.985170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 149/306] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:32.985850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0259e-6bc7-426d-b4da-dbbfce23d8b1 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:32.996130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0259e-6bc7-426d-b4da-dbbfce23d8b1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:32.996130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0259e-6bc7-426d-b4da-dbbfce23d8b1 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:32.996130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 150/307] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:32.996130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0259e-6bc7-426d-b4da-dbbfce23d8b1 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:33.071986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38efaf28-4f3e-47d9-a159-a80b35e5b3f7 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:33.083815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:33.122440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0259e-6bc7-426d-b4da-dbbfce23d8b1 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:33.126534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4f7de-c472-46da-8d71-da86c9532b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:33.127217 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4f7de-c472-46da-8d71-da86c9532b3f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:33.127506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4f7de-c472-46da-8d71-da86c9532b3f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:33.127788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4f7de-c472-46da-8d71-da86c9532b3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:33.128716 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4f7de-c472-46da-8d71-da86c9532b3f None admin] RBAC: Adding query filter params (role.id=84bd13e4cee04c12bd6a0e549db79246, user.id=5d81d732fa4148edacdb6aaaefa81cdc, scope.system=all) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:33.129043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4f7de-c472-46da-8d71-da86c9532b3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:33.130990 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4f7de-c472-46da-8d71-da86c9532b3f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:33.137657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 158/308] 10.222.0.64 () {56 vars in 1308 bytes} [Thu Jul 14 17:37:32 2022] GET /identity/v3/role_assignments?user.id=5d81d732fa4148edacdb6aaaefa81cdc&scope.system=all&role.id=84bd13e4cee04c12bd6a0e549db79246 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:33.138662 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d759d94-82b5-4ac1-963b-738f4d4941bf None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:33.138957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d759d94-82b5-4ac1-963b-738f4d4941bf None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:33.139237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d759d94-82b5-4ac1-963b-738f4d4941bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:33.158089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-7d759d94-82b5-4ac1-963b-738f4d4941bf None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:33.159354 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7d759d94-82b5-4ac1-963b-738f4d4941bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:33.164294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759043e-951d-4c13-a479-b9f2f2172d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:33.165799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759043e-951d-4c13-a479-b9f2f2172d40 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:33.165799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759043e-951d-4c13-a479-b9f2f2172d40 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:33.165799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759043e-951d-4c13-a479-b9f2f2172d40 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:33.166286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759043e-951d-4c13-a479-b9f2f2172d40 None admin] RBAC: Adding query filter params (name=member) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:33.166595 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759043e-951d-4c13-a479-b9f2f2172d40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:33.168085 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759043e-951d-4c13-a479-b9f2f2172d40 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:33.179855 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 151/309] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:33.186617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759043e-951d-4c13-a479-b9f2f2172d40 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:33.205941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:33.252052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 159/310] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:33.271310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d759d94-82b5-4ac1-963b-738f4d4941bf None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:33.318837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759043e-951d-4c13-a479-b9f2f2172d40 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:33.356099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820135b1-5c30-431c-b4ee-eaa04142feed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:33.357066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820135b1-5c30-431c-b4ee-eaa04142feed None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:33.357453 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820135b1-5c30-431c-b4ee-eaa04142feed None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:33.357775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820135b1-5c30-431c-b4ee-eaa04142feed None admin] PATH_INFO: `/v3/users/d81458a88938477092806576f4b080ff` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:33.359147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820135b1-5c30-431c-b4ee-eaa04142feed None admin] RBAC: Authorizing `identity:get_user(user_id=d81458a88938477092806576f4b080ff)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:33.360749 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820135b1-5c30-431c-b4ee-eaa04142feed None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:33.367184 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 152/311] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:33 2022] GET /identity/v3/users/d81458a88938477092806576f4b080ff => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:33.368901 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820135b1-5c30-431c-b4ee-eaa04142feed None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:33.382101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d759d94-82b5-4ac1-963b-738f4d4941bf None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:33.384622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:33.421365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80926287-69f3-429a-8133-c37bf7f4ffd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:33.422089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80926287-69f3-429a-8133-c37bf7f4ffd5 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:33.422376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80926287-69f3-429a-8133-c37bf7f4ffd5 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:33.422651 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80926287-69f3-429a-8133-c37bf7f4ffd5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:33.438382 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80926287-69f3-429a-8133-c37bf7f4ffd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-80926287-69f3-429a-8133-c37bf7f4ffd5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:33.450306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:33.451241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:33.452060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:33.452060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:33.452060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:33.452060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:33.452060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:33.452060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:33.452060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:33.452060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:33.452060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:33.452060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:33.452060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:33.452060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:33.458222 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 160/312] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:33.460096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:33.460500 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80926287-69f3-429a-8133-c37bf7f4ffd5 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:33.504361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820135b1-5c30-431c-b4ee-eaa04142feed None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:33.546278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bf6d3-d116-426a-903a-87f5726f9fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:33.547345 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bf6d3-d116-426a-903a-87f5726f9fbb None admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:33.547629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bf6d3-d116-426a-903a-87f5726f9fbb None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:33.547908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013bf6d3-d116-426a-903a-87f5726f9fbb None admin] PATH_INFO: `/v3/system/users/d81458a88938477092806576f4b080ff/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:33.550064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bf6d3-d116-426a-903a-87f5726f9fbb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d81458a88938477092806576f4b080ff, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:33.551631 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013bf6d3-d116-426a-903a-87f5726f9fbb None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:33.564727 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 153/313] 10.222.0.64 () {58 vars in 1334 bytes} [Thu Jul 14 17:37:33 2022] PUT /identity/v3/system/users/d81458a88938477092806576f4b080ff/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:37:33.603017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80926287-69f3-429a-8133-c37bf7f4ffd5 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:33.644394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752dcae-7a85-4535-b08c-638a57e56888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:33.645476 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752dcae-7a85-4535-b08c-638a57e56888 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:33.645971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752dcae-7a85-4535-b08c-638a57e56888 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:33.646541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c752dcae-7a85-4535-b08c-638a57e56888 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:33.647570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752dcae-7a85-4535-b08c-638a57e56888 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:33.648112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752dcae-7a85-4535-b08c-638a57e56888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:33.649824 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c752dcae-7a85-4535-b08c-638a57e56888 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:33.660973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 161/314] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:33.668088 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bf6d3-d116-426a-903a-87f5726f9fbb None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:33.782713 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013bf6d3-d116-426a-903a-87f5726f9fbb None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:33.820297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ffedb-848b-4eff-90b9-f434ed8423e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:33.821540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ffedb-848b-4eff-90b9-f434ed8423e4 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:33.822029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ffedb-848b-4eff-90b9-f434ed8423e4 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:33.822534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ffedb-848b-4eff-90b9-f434ed8423e4 None admin] PATH_INFO: `/v3/users/0c859050ce6740fc8ee0f081c5b4e9e7` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:33.824630 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ffedb-848b-4eff-90b9-f434ed8423e4 None admin] RBAC: Authorizing `identity:get_user(user_id=0c859050ce6740fc8ee0f081c5b4e9e7)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:33.826407 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ffedb-848b-4eff-90b9-f434ed8423e4 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:33.828401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 154/315] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:33 2022] GET /identity/v3/users/0c859050ce6740fc8ee0f081c5b4e9e7 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:33.834281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752dcae-7a85-4535-b08c-638a57e56888 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:33.865972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:33.948176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c752dcae-7a85-4535-b08c-638a57e56888 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:33.984661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8bacf-cd43-41b6-b82d-e5f6ba2fa669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:33.985320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8bacf-cd43-41b6-b82d-e5f6ba2fa669 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:33.985605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8bacf-cd43-41b6-b82d-e5f6ba2fa669 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:33.985883 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e8bacf-cd43-41b6-b82d-e5f6ba2fa669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:34.059185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8bacf-cd43-41b6-b82d-e5f6ba2fa669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:34.060616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e8bacf-cd43-41b6-b82d-e5f6ba2fa669 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:34.088207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:34.170428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 162/316] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:34.176279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ffedb-848b-4eff-90b9-f434ed8423e4 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:34.179563 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:34.302299 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ffedb-848b-4eff-90b9-f434ed8423e4 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:34.338461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add2567-0c7c-4480-bda8-5f9e25f12028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:34.339326 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add2567-0c7c-4480-bda8-5f9e25f12028 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:34.340129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add2567-0c7c-4480-bda8-5f9e25f12028 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:34.340410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add2567-0c7c-4480-bda8-5f9e25f12028 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:34.341609 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add2567-0c7c-4480-bda8-5f9e25f12028 None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:34.343062 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add2567-0c7c-4480-bda8-5f9e25f12028 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:34.345107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 155/317] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:34 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:34.353072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8bacf-cd43-41b6-b82d-e5f6ba2fa669 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:34.463802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e8bacf-cd43-41b6-b82d-e5f6ba2fa669 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:34.490474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:34.500240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de1005-0cab-446f-aa14-f1accf810076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:34.500905 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de1005-0cab-446f-aa14-f1accf810076 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:34.501193 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de1005-0cab-446f-aa14-f1accf810076 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:34.501474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de1005-0cab-446f-aa14-f1accf810076 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:34.502311 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de1005-0cab-446f-aa14-f1accf810076 None admin] RBAC: Adding query filter params (role.id=84bd13e4cee04c12bd6a0e549db79246, scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2, user.id=0c859050ce6740fc8ee0f081c5b4e9e7) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:34.502634 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de1005-0cab-446f-aa14-f1accf810076 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:34.504101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de1005-0cab-446f-aa14-f1accf810076 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:34.510577 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 163/318] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:37:34 2022] GET /identity/v3/role_assignments?user.id=0c859050ce6740fc8ee0f081c5b4e9e7&scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2&role.id=84bd13e4cee04c12bd6a0e549db79246 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:35.180367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ada99f-d821-4c75-b0aa-073399d89fd5 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:35.180688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ada99f-d821-4c75-b0aa-073399d89fd5 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:35.180938 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ada99f-d821-4c75-b0aa-073399d89fd5 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:35.182343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 156/319] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:35.188282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b858e-7146-43de-be01-82b244fab9e3 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:35.188575 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b858e-7146-43de-be01-82b244fab9e3 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:35.188858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8b858e-7146-43de-be01-82b244fab9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:35.206186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-dd8b858e-7146-43de-be01-82b244fab9e3 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:35.207392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-dd8b858e-7146-43de-be01-82b244fab9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:35.224622 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a356e-0fbd-4329-ad7b-0f9ac5c4e057 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:35.224894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a356e-0fbd-4329-ad7b-0f9ac5c4e057 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:35.225342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a356e-0fbd-4329-ad7b-0f9ac5c4e057 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:35.226705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 157/320] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:37:35.232038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927721f-9534-4c3a-a0a1-602e42cd3e81 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:35.232369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927721f-9534-4c3a-a0a1-602e42cd3e81 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:35.232658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5927721f-9534-4c3a-a0a1-602e42cd3e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:35.250115 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5927721f-9534-4c3a-a0a1-602e42cd3e81 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:35.251336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5927721f-9534-4c3a-a0a1-602e42cd3e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:35.278640 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 164/321] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:35.301319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b858e-7146-43de-be01-82b244fab9e3 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:35.318014 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:35.324661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 158/322] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:35.346755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927721f-9534-4c3a-a0a1-602e42cd3e81 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:35.412247 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8b858e-7146-43de-be01-82b244fab9e3 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:35.436437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:35.451908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5927721f-9534-4c3a-a0a1-602e42cd3e81 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:35.452143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4578b-93f7-42b4-bfcc-695a17e9898e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:35.452888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4578b-93f7-42b4-bfcc-695a17e9898e None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:35.453171 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4578b-93f7-42b4-bfcc-695a17e9898e None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:35.453444 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4578b-93f7-42b4-bfcc-695a17e9898e None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:35.454868 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4578b-93f7-42b4-bfcc-695a17e9898e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:35.456394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e4578b-93f7-42b4-bfcc-695a17e9898e None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:35.460395 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 165/323] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:37:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:35.463496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4578b-93f7-42b4-bfcc-695a17e9898e None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:35.490470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d8de4-cddc-4245-81d4-3a5ec3ade96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:35.491537 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d8de4-cddc-4245-81d4-3a5ec3ade96f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:35.491989 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d8de4-cddc-4245-81d4-3a5ec3ade96f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:35.492398 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d8de4-cddc-4245-81d4-3a5ec3ade96f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:35.503870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d8de4-cddc-4245-81d4-3a5ec3ade96f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-bc8d8de4-cddc-4245-81d4-3a5ec3ade96f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:35.512372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:35.513527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:35.514503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:35.514503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:35.514503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:35.514503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:35.514503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:35.514503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:35.514503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:35.514503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:35.514503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:35.514503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:35.514503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:35.514503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:35.516818 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 159/324] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:35.522292 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d8de4-cddc-4245-81d4-3a5ec3ade96f None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:35.577079 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4578b-93f7-42b4-bfcc-695a17e9898e None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:35.591290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:35.613395 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f8540-66ad-49c5-b442-716995aab4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:35.614074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f8540-66ad-49c5-b442-716995aab4c0 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:35.614356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f8540-66ad-49c5-b442-716995aab4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:35.614632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f8540-66ad-49c5-b442-716995aab4c0 None admin] PATH_INFO: `/v3/groups` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:35.615321 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f8540-66ad-49c5-b442-716995aab4c0 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:35.616826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f8540-66ad-49c5-b442-716995aab4c0 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:35.631965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 166/325] 10.222.0.64 () {60 vars in 1134 bytes} [Thu Jul 14 17:37:35 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:37:35.644119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d8de4-cddc-4245-81d4-3a5ec3ade96f None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:35.683802 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a530e8c2-74c6-4284-99f9-38b6c785f477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:35.684468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530e8c2-74c6-4284-99f9-38b6c785f477 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:35.684758 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530e8c2-74c6-4284-99f9-38b6c785f477 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:35.685044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530e8c2-74c6-4284-99f9-38b6c785f477 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:35.685933 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530e8c2-74c6-4284-99f9-38b6c785f477 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:35.686259 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530e8c2-74c6-4284-99f9-38b6c785f477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:35.687839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a530e8c2-74c6-4284-99f9-38b6c785f477 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:35.703499 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 160/326] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:35.704688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f8540-66ad-49c5-b442-716995aab4c0 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:35.763194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:35.764487 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b18d6-55ac-4bdd-968f-06968a9a1a05 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:35.764646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b18d6-55ac-4bdd-968f-06968a9a1a05 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:35.764874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b18d6-55ac-4bdd-968f-06968a9a1a05 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:35.766357 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 161/327] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:37:35.772667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3ff0b-d1aa-4fdc-8d90-4e6db56186f7 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:35.773136 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3ff0b-d1aa-4fdc-8d90-4e6db56186f7 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:35.773399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3ff0b-d1aa-4fdc-8d90-4e6db56186f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:35.791174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-eed3ff0b-d1aa-4fdc-8d90-4e6db56186f7 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:35.792990 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-eed3ff0b-d1aa-4fdc-8d90-4e6db56186f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:35.827031 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f8540-66ad-49c5-b442-716995aab4c0 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:35.864848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b688c10-779b-40c4-8cde-38786c751a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:35.865749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b688c10-779b-40c4-8cde-38786c751a11 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:35.866040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b688c10-779b-40c4-8cde-38786c751a11 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:35.866319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b688c10-779b-40c4-8cde-38786c751a11 None admin] PATH_INFO: `/v3/users/5d81d732fa4148edacdb6aaaefa81cdc` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:35.867528 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b688c10-779b-40c4-8cde-38786c751a11 None admin] RBAC: Authorizing `identity:get_user(user_id=5d81d732fa4148edacdb6aaaefa81cdc)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:35.869027 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b688c10-779b-40c4-8cde-38786c751a11 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:35.870319 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 162/328] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:35.871608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 167/329] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:35 2022] GET /identity/v3/users/5d81d732fa4148edacdb6aaaefa81cdc => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:35.877323 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3ff0b-d1aa-4fdc-8d90-4e6db56186f7 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:35.895345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b688c10-779b-40c4-8cde-38786c751a11 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:35.945802 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:36.008368 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3ff0b-d1aa-4fdc-8d90-4e6db56186f7 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:36.014425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b688c10-779b-40c4-8cde-38786c751a11 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:36.035477 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:36.058671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048c0c3-2544-409c-a972-cef3bf8dc112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:36.059627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048c0c3-2544-409c-a972-cef3bf8dc112 None admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:36.059917 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048c0c3-2544-409c-a972-cef3bf8dc112 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:36.060245 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2048c0c3-2544-409c-a972-cef3bf8dc112 None admin] PATH_INFO: `/v3/system/users/5d81d732fa4148edacdb6aaaefa81cdc/roles/84bd13e4cee04c12bd6a0e549db79246` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:36.061578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe3c6e-c7c2-46a1-8bd7-71c395af603c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:36.061857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2048c0c3-2544-409c-a972-cef3bf8dc112 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5d81d732fa4148edacdb6aaaefa81cdc, role_id=84bd13e4cee04c12bd6a0e549db79246)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:36.062612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe3c6e-c7c2-46a1-8bd7-71c395af603c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:36.062896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe3c6e-c7c2-46a1-8bd7-71c395af603c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:36.063173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe3c6e-c7c2-46a1-8bd7-71c395af603c None admin] PATH_INFO: `/v3/roles/member` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:36.063337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2048c0c3-2544-409c-a972-cef3bf8dc112 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:36.076384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 163/330] 10.222.0.64 () {58 vars in 1334 bytes} [Thu Jul 14 17:37:35 2022] PUT /identity/v3/system/users/5d81d732fa4148edacdb6aaaefa81cdc/roles/84bd13e4cee04c12bd6a0e549db79246 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:37:36.085168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe3c6e-c7c2-46a1-8bd7-71c395af603c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-ddfe3c6e-c7c2-46a1-8bd7-71c395af603c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:36.093750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:36.094704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:36.095487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:36.095487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:36.095487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:36.095487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:36.095487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:36.095487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:36.095487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:36.095487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:36.095487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:36.095487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:36.095487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:36.095487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:36.097660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 168/331] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:36.104099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048c0c3-2544-409c-a972-cef3bf8dc112 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:36.232497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2048c0c3-2544-409c-a972-cef3bf8dc112 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:36.252939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:36.282339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1b79e-424b-4c44-ae87-e6524550f192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:36.283035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1b79e-424b-4c44-ae87-e6524550f192 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:36.283319 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1b79e-424b-4c44-ae87-e6524550f192 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:36.283598 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1b79e-424b-4c44-ae87-e6524550f192 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:36.284495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1b79e-424b-4c44-ae87-e6524550f192 None admin] RBAC: Adding query filter params (name=member) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:36.284808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1b79e-424b-4c44-ae87-e6524550f192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:36.286215 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1b79e-424b-4c44-ae87-e6524550f192 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:36.298418 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 164/332] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:36.304570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe3c6e-c7c2-46a1-8bd7-71c395af603c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:36.430287 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe3c6e-c7c2-46a1-8bd7-71c395af603c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:36.485475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae71172-c021-40a3-aa2a-400709ab9140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:36.486304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae71172-c021-40a3-aa2a-400709ab9140 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:36.486592 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae71172-c021-40a3-aa2a-400709ab9140 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:36.486871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae71172-c021-40a3-aa2a-400709ab9140 None admin] PATH_INFO: `/v3/users/d81458a88938477092806576f4b080ff` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:36.492725 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae71172-c021-40a3-aa2a-400709ab9140 None admin] RBAC: Authorizing `identity:get_user(user_id=d81458a88938477092806576f4b080ff)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:36.494208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae71172-c021-40a3-aa2a-400709ab9140 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:36.496949 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 169/333] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:36 2022] GET /identity/v3/users/d81458a88938477092806576f4b080ff => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:36.502592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1b79e-424b-4c44-ae87-e6524550f192 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:36.638328 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1b79e-424b-4c44-ae87-e6524550f192 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:36.679613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8cdd5d-57d2-4614-9514-eb9e3f83704c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:36.680336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cdd5d-57d2-4614-9514-eb9e3f83704c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:36.680626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cdd5d-57d2-4614-9514-eb9e3f83704c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:36.680908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8cdd5d-57d2-4614-9514-eb9e3f83704c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:36.681773 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cdd5d-57d2-4614-9514-eb9e3f83704c None admin] RBAC: Adding query filter params (role.id=fbd0f348273f4791b1813d1df24df10e, user.id=d81458a88938477092806576f4b080ff, scope.system=all) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:36.682091 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cdd5d-57d2-4614-9514-eb9e3f83704c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:36.683615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8cdd5d-57d2-4614-9514-eb9e3f83704c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:36.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 165/334] 10.222.0.64 () {56 vars in 1308 bytes} [Thu Jul 14 17:37:36 2022] GET /identity/v3/role_assignments?user.id=d81458a88938477092806576f4b080ff&scope.system=all&role.id=fbd0f348273f4791b1813d1df24df10e => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:38.224376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb78100-d7f7-48ab-a13f-9e566268a73a None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:38.225393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb78100-d7f7-48ab-a13f-9e566268a73a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:38.225921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb78100-d7f7-48ab-a13f-9e566268a73a None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:38.227806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 170/335] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:38.232970 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51b7b4-9428-4c8f-9ce6-bf1084f645e4 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:38.233827 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51b7b4-9428-4c8f-9ce6-bf1084f645e4 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:38.234309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c51b7b4-9428-4c8f-9ce6-bf1084f645e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:38.258747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-1c51b7b4-9428-4c8f-9ce6-bf1084f645e4 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:38.260344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1c51b7b4-9428-4c8f-9ce6-bf1084f645e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:38.299950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:38.345690 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 166/336] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:38.396292 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb78100-d7f7-48ab-a13f-9e566268a73a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:38.458393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:38.524734 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb78100-d7f7-48ab-a13f-9e566268a73a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:38.573118 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03247dd2-62a0-4261-b74f-248aade0f5e7 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:38.574389 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03247dd2-62a0-4261-b74f-248aade0f5e7 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:38.575008 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03247dd2-62a0-4261-b74f-248aade0f5e7 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:38.576812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 167/337] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:37:38.582533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33ab9f-711b-4b7a-99af-af340711d992 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:38.582921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33ab9f-711b-4b7a-99af-af340711d992 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:38.583089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e33ab9f-711b-4b7a-99af-af340711d992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:38.584892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56219a02-d4be-4a50-a681-420e7e5e5968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:38.585716 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56219a02-d4be-4a50-a681-420e7e5e5968 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:38.586004 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56219a02-d4be-4a50-a681-420e7e5e5968 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:38.586281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56219a02-d4be-4a50-a681-420e7e5e5968 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:38.600089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56219a02-d4be-4a50-a681-420e7e5e5968 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-56219a02-d4be-4a50-a681-420e7e5e5968 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:38.609360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:38.610406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:38.611212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:38.611212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:38.611212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:38.611212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:38.611212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:38.611212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:38.611212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:38.611212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:38.611212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:38.611212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:38.611212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:38.611212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:38.614193 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 171/338] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:38.620013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-1e33ab9f-711b-4b7a-99af-af340711d992 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:38.621804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1e33ab9f-711b-4b7a-99af-af340711d992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:38.622040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56219a02-d4be-4a50-a681-420e7e5e5968 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:38.742454 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 168/339] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:38.789694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56219a02-d4be-4a50-a681-420e7e5e5968 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:38.804296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:38.833056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33ab9f-711b-4b7a-99af-af340711d992 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:38.859452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478953d-3cd7-456a-a592-4fb3d4f8fb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:38.860170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478953d-3cd7-456a-a592-4fb3d4f8fb74 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:38.860456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478953d-3cd7-456a-a592-4fb3d4f8fb74 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:38.860732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478953d-3cd7-456a-a592-4fb3d4f8fb74 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:38.861499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478953d-3cd7-456a-a592-4fb3d4f8fb74 None admin] RBAC: Adding query filter params (name=member) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:38.861810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478953d-3cd7-456a-a592-4fb3d4f8fb74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:38.863207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478953d-3cd7-456a-a592-4fb3d4f8fb74 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:38.878509 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 172/340] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:38.884169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478953d-3cd7-456a-a592-4fb3d4f8fb74 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:38.985227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:38.997670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e33ab9f-711b-4b7a-99af-af340711d992 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:39.090132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478953d-3cd7-456a-a592-4fb3d4f8fb74 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:39.111734 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f8eb1-dc9d-4762-a0e6-bfb89fccdf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:39.113697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f8eb1-dc9d-4762-a0e6-bfb89fccdf30 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:39.114486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f8eb1-dc9d-4762-a0e6-bfb89fccdf30 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:39.124728 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f8eb1-dc9d-4762-a0e6-bfb89fccdf30 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:39.161483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f8eb1-dc9d-4762-a0e6-bfb89fccdf30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:39.165316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d88f61-093e-4f68-83a5-f54a197eb1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:39.165996 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d88f61-093e-4f68-83a5-f54a197eb1a4 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:39.166285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d88f61-093e-4f68-83a5-f54a197eb1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:39.166568 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d88f61-093e-4f68-83a5-f54a197eb1a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-6d9f8eb1-dc9d-4762-a0e6-bfb89fccdf30 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:39.182800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:39.185100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:39.186626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:39.186626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:39.186626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:39.186626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:39.186626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:39.186626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:39.186626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:39.186626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:39.186626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:39.186626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:39.186626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:39.186626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:39.191607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 169/341] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:39.198385 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f8eb1-dc9d-4762-a0e6-bfb89fccdf30 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:39.222511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:39.274276 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d88f61-093e-4f68-83a5-f54a197eb1a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:39.275663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d88f61-093e-4f68-83a5-f54a197eb1a4 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:39.430548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f8eb1-dc9d-4762-a0e6-bfb89fccdf30 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:39.508500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d3dcc-2ba0-4d3a-a08a-6196cac0f469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:39.510152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d3dcc-2ba0-4d3a-a08a-6196cac0f469 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:39.510942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d3dcc-2ba0-4d3a-a08a-6196cac0f469 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:39.511750 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d3dcc-2ba0-4d3a-a08a-6196cac0f469 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:39.513505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d3dcc-2ba0-4d3a-a08a-6196cac0f469 None admin] RBAC: Adding query filter params (name=member) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:39.516565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d3dcc-2ba0-4d3a-a08a-6196cac0f469 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:39.519323 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d3dcc-2ba0-4d3a-a08a-6196cac0f469 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:39.548494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 173/342] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 665 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:39.555562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 170/343] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:39.562070 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d88f61-093e-4f68-83a5-f54a197eb1a4 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:39.574956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d3dcc-2ba0-4d3a-a08a-6196cac0f469 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:39.636774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:39.748346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:39.830064 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d88f61-093e-4f68-83a5-f54a197eb1a4 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:39.856303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d3dcc-2ba0-4d3a-a08a-6196cac0f469 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:39.907231 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a42d59-b180-4de3-b3f3-f686193b43ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:39.908144 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a42d59-b180-4de3-b3f3-f686193b43ea None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:39.908435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a42d59-b180-4de3-b3f3-f686193b43ea None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:39.908720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a42d59-b180-4de3-b3f3-f686193b43ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:39.909837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33eff7-c201-481c-b12a-081b570222a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:39.911312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33eff7-c201-481c-b12a-081b570222a0 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:39.912210 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33eff7-c201-481c-b12a-081b570222a0 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:39.912625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33eff7-c201-481c-b12a-081b570222a0 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:39.916245 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33eff7-c201-481c-b12a-081b570222a0 None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:39.918942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33eff7-c201-481c-b12a-081b570222a0 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:39.924489 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 171/344] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:39 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:39.926771 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2198b0-d860-4720-a38a-6096af2a256c None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:39.927238 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2198b0-d860-4720-a38a-6096af2a256c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:39.927700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2198b0-d860-4720-a38a-6096af2a256c None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:39.932304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 172/345] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:39.935319 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c7e0e-cd07-478c-91ba-8d124f37e2c8 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:39.936662 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c7e0e-cd07-478c-91ba-8d124f37e2c8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:39.937096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6c7e0e-cd07-478c-91ba-8d124f37e2c8 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:39.939787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 173/346] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:39.941403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c7e0e-cd07-478c-91ba-8d124f37e2c8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:40.047128 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a42d59-b180-4de3-b3f3-f686193b43ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:40.052118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a42d59-b180-4de3-b3f3-f686193b43ea None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:40.078786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:40.109207 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6c7e0e-cd07-478c-91ba-8d124f37e2c8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:40.159002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca4116-443b-44b9-a968-904b484449d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:40.159915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca4116-443b-44b9-a968-904b484449d8 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:40.160483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca4116-443b-44b9-a968-904b484449d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:40.160972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca4116-443b-44b9-a968-904b484449d8 None admin] PATH_INFO: `/v3/groups/50fa9d01d91a423bb20869ca7519fea0` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:40.166800 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:40.170161 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca4116-443b-44b9-a968-904b484449d8 None admin] RBAC: Authorizing `identity:get_group(group_id=50fa9d01d91a423bb20869ca7519fea0)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:40.171765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ca4116-443b-44b9-a968-904b484449d8 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:40.173927 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 174/347] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:39 2022] GET /identity/v3/groups/50fa9d01d91a423bb20869ca7519fea0 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:40.180405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fdb21-3c14-48a3-a9a5-b252007136ff None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:40.180644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fdb21-3c14-48a3-a9a5-b252007136ff None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:40.180919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fdb21-3c14-48a3-a9a5-b252007136ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:40.204029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-6b5fdb21-3c14-48a3-a9a5-b252007136ff None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:40.205341 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6b5fdb21-3c14-48a3-a9a5-b252007136ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:40.228822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 174/348] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:40.230302 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c15ac-8ca6-49b3-9c5d-79f14ad53383 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:40.230585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c15ac-8ca6-49b3-9c5d-79f14ad53383 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:40.230869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c15ac-8ca6-49b3-9c5d-79f14ad53383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:40.253113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-434c15ac-8ca6-49b3-9c5d-79f14ad53383 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:40.254483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-434c15ac-8ca6-49b3-9c5d-79f14ad53383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:40.315734 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 175/349] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:40.317254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2be2b-3361-4fc4-a218-cb309f500796 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:40.317546 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2be2b-3361-4fc4-a218-cb309f500796 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:40.317891 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2be2b-3361-4fc4-a218-cb309f500796 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:40.319557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 176/350] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:40.320153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2be2b-3361-4fc4-a218-cb309f500796 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:40.379067 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 175/351] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:40.379764 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c15ac-8ca6-49b3-9c5d-79f14ad53383 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:40.463101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:40.504865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2be2b-3361-4fc4-a218-cb309f500796 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:40.573428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742bef8-7320-4f62-bd8f-c813bcf6e756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:40.574552 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742bef8-7320-4f62-bd8f-c813bcf6e756 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:40.575045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742bef8-7320-4f62-bd8f-c813bcf6e756 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:40.575596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742bef8-7320-4f62-bd8f-c813bcf6e756 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:40.577294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742bef8-7320-4f62-bd8f-c813bcf6e756 None admin] RBAC: Adding query filter params (group.id=50fa9d01d91a423bb20869ca7519fea0, role.id=fbd0f348273f4791b1813d1df24df10e, scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:40.577832 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742bef8-7320-4f62-bd8f-c813bcf6e756 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:40.581072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742bef8-7320-4f62-bd8f-c813bcf6e756 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:40.591548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 177/352] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Jul 14 17:37:40 2022] GET /identity/v3/role_assignments?group.id=50fa9d01d91a423bb20869ca7519fea0&scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82&role.id=fbd0f348273f4791b1813d1df24df10e => generated 260 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:40.593851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2151986-d04c-4e5a-a145-7679e2c7ecfd None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:40.594351 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2151986-d04c-4e5a-a145-7679e2c7ecfd None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:40.594816 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2151986-d04c-4e5a-a145-7679e2c7ecfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:40.604483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:40.613729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c15ac-8ca6-49b3-9c5d-79f14ad53383 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:40.624568 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-c2151986-d04c-4e5a-a145-7679e2c7ecfd None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:40.626278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c2151986-d04c-4e5a-a145-7679e2c7ecfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:40.694233 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a831d8-fb10-4db6-bcfb-2e4166f64d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:40.695819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a831d8-fb10-4db6-bcfb-2e4166f64d20 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:40.696363 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a831d8-fb10-4db6-bcfb-2e4166f64d20 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:40.696845 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a831d8-fb10-4db6-bcfb-2e4166f64d20 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:40.700311 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a831d8-fb10-4db6-bcfb-2e4166f64d20 None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:40.702842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a831d8-fb10-4db6-bcfb-2e4166f64d20 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:40.706507 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 176/353] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:40 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:40.707828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a831d8-fb10-4db6-bcfb-2e4166f64d20 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:40.719058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 178/354] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:40.722494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2151986-d04c-4e5a-a145-7679e2c7ecfd None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:40.882369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:40.909803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a831d8-fb10-4db6-bcfb-2e4166f64d20 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:40.943527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2151986-d04c-4e5a-a145-7679e2c7ecfd None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:40.981282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6964c5e6-b874-4dc9-8a92-44a052aaa220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:40.982327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964c5e6-b874-4dc9-8a92-44a052aaa220 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:40.982854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964c5e6-b874-4dc9-8a92-44a052aaa220 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:40.983303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6964c5e6-b874-4dc9-8a92-44a052aaa220 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:40.995333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac6afa-1236-41f1-aeb6-e92ad2835d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:40.997380 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac6afa-1236-41f1-aeb6-e92ad2835d0d None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:40.997888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac6afa-1236-41f1-aeb6-e92ad2835d0d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:40.998384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac6afa-1236-41f1-aeb6-e92ad2835d0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:41.007458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6964c5e6-b874-4dc9-8a92-44a052aaa220 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:41.019639 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac6afa-1236-41f1-aeb6-e92ad2835d0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-6964c5e6-b874-4dc9-8a92-44a052aaa220 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:41.020537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:41.021428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:41.022226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:41.022226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:41.022226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:41.022226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:41.022226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:41.022226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:41.022226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:41.022226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:41.022226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:41.022226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:41.022226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 17:37:41.022226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:41.022829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 177/355] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:41.025072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1892bb3-458b-4d6c-bd27-6986ea58c978 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:41.025546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1892bb3-458b-4d6c-bd27-6986ea58c978 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:41.026011 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1892bb3-458b-4d6c-bd27-6986ea58c978 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:41.029080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 178/356] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:41.030443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1892bb3-458b-4d6c-bd27-6986ea58c978 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-37ac6afa-1236-41f1-aeb6-e92ad2835d0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:41.036410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:41.037578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:41.039531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:41.039531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:41.039531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:41.039531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:41.039531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:41.039531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:41.039531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:41.039531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:41.039531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:41.039531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:41.039531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:41.039531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:41.039531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 179/357] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:37:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:41.041424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac6afa-1236-41f1-aeb6-e92ad2835d0d None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:41.081664 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:41.248148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1892bb3-458b-4d6c-bd27-6986ea58c978 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:41.261785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac6afa-1236-41f1-aeb6-e92ad2835d0d None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:41.314688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492a17e-a0cb-4244-baf7-dd55b5c6808d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:41.315581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492a17e-a0cb-4244-baf7-dd55b5c6808d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:41.315867 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492a17e-a0cb-4244-baf7-dd55b5c6808d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:41.316180 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492a17e-a0cb-4244-baf7-dd55b5c6808d None admin] PATH_INFO: `/v3/groups/50fa9d01d91a423bb20869ca7519fea0` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:41.317865 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492a17e-a0cb-4244-baf7-dd55b5c6808d None admin] RBAC: Authorizing `identity:get_group(group_id=50fa9d01d91a423bb20869ca7519fea0)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:41.319346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492a17e-a0cb-4244-baf7-dd55b5c6808d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:41.321183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 179/358] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:41 2022] GET /identity/v3/groups/50fa9d01d91a423bb20869ca7519fea0 => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:41.324852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492a17e-a0cb-4244-baf7-dd55b5c6808d None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:41.327421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9540e59-4939-48b6-bb4b-dbadabcc3dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:41.329240 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9540e59-4939-48b6-bb4b-dbadabcc3dfa None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:41.329964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9540e59-4939-48b6-bb4b-dbadabcc3dfa None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:41.330437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9540e59-4939-48b6-bb4b-dbadabcc3dfa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:41.333396 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:41.353146 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9540e59-4939-48b6-bb4b-dbadabcc3dfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-d9540e59-4939-48b6-bb4b-dbadabcc3dfa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:41.368885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:41.370243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:41.371287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:41.371287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:41.371287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:41.371287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:41.371287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:41.371287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:41.371287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:41.371287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:41.371287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:41.371287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:41.371287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:41.371287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:41.371980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 180/359] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:37:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:41.373712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865100b2-7554-4205-8963-aad38bd1f6e8 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:41.374210 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865100b2-7554-4205-8963-aad38bd1f6e8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:41.374686 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865100b2-7554-4205-8963-aad38bd1f6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:41.406374 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-865100b2-7554-4205-8963-aad38bd1f6e8 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:41.408387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-865100b2-7554-4205-8963-aad38bd1f6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:41.496965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:41.508163 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492a17e-a0cb-4244-baf7-dd55b5c6808d None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:41.533767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 181/360] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:41.535179 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865100b2-7554-4205-8963-aad38bd1f6e8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:41.582199 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4eac23-69b2-48d6-96c9-8f6401b27e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:41.584589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4eac23-69b2-48d6-96c9-8f6401b27e52 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:41.585253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4eac23-69b2-48d6-96c9-8f6401b27e52 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:41.586198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4eac23-69b2-48d6-96c9-8f6401b27e52 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:41.587803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4eac23-69b2-48d6-96c9-8f6401b27e52 None admin] RBAC: Adding query filter params (name=reader) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:41.588797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4eac23-69b2-48d6-96c9-8f6401b27e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:41.592725 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4eac23-69b2-48d6-96c9-8f6401b27e52 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:41.616830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 180/361] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:41.618927 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4eac23-69b2-48d6-96c9-8f6401b27e52 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:41.744661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865100b2-7554-4205-8963-aad38bd1f6e8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:41.793786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:41.803129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4eac23-69b2-48d6-96c9-8f6401b27e52 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:41.807193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4245490-7100-45e0-b225-8a58618fe76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:41.807873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4245490-7100-45e0-b225-8a58618fe76c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:41.808849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4245490-7100-45e0-b225-8a58618fe76c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:41.809122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4245490-7100-45e0-b225-8a58618fe76c None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:41.809911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4245490-7100-45e0-b225-8a58618fe76c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:41.810221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4245490-7100-45e0-b225-8a58618fe76c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:41.811639 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4245490-7100-45e0-b225-8a58618fe76c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:41.821346 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 182/362] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:37:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:41.822063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4245490-7100-45e0-b225-8a58618fe76c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:41.866660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c584fe-1c23-4634-ab82-adeed5143b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:41.867368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c584fe-1c23-4634-ab82-adeed5143b0f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:41.867670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c584fe-1c23-4634-ab82-adeed5143b0f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:41.868020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c584fe-1c23-4634-ab82-adeed5143b0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:41.868935 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c584fe-1c23-4634-ab82-adeed5143b0f None admin] RBAC: Adding query filter params (group.id=50fa9d01d91a423bb20869ca7519fea0, role.id=fbd0f348273f4791b1813d1df24df10e, scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:41.869738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c584fe-1c23-4634-ab82-adeed5143b0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:41.871844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c584fe-1c23-4634-ab82-adeed5143b0f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:41.880856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 181/363] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Jul 14 17:37:41 2022] GET /identity/v3/role_assignments?group.id=50fa9d01d91a423bb20869ca7519fea0&scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2&role.id=fbd0f348273f4791b1813d1df24df10e => generated 260 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:41.881867 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c584fe-1c23-4634-ab82-adeed5143b0f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:41.932522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:42.002307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4245490-7100-45e0-b225-8a58618fe76c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:42.049334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e11d9-dcf0-47c3-bff7-36d2d1cb1c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:42.049921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e11d9-dcf0-47c3-bff7-36d2d1cb1c2f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:42.050208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e11d9-dcf0-47c3-bff7-36d2d1cb1c2f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:42.050493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e11d9-dcf0-47c3-bff7-36d2d1cb1c2f None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:42.051269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e11d9-dcf0-47c3-bff7-36d2d1cb1c2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:42.051580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e11d9-dcf0-47c3-bff7-36d2d1cb1c2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:42.053075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e11d9-dcf0-47c3-bff7-36d2d1cb1c2f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:42.065594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 183/364] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:37:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:42.066324 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e11d9-dcf0-47c3-bff7-36d2d1cb1c2f None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:42.097744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c584fe-1c23-4634-ab82-adeed5143b0f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:42.158936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621d438-40cd-4457-a8fa-4f5730dfb6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:42.160771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621d438-40cd-4457-a8fa-4f5730dfb6ec None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:42.161399 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621d438-40cd-4457-a8fa-4f5730dfb6ec None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:42.161891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621d438-40cd-4457-a8fa-4f5730dfb6ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:42.176955 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:42.178169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621d438-40cd-4457-a8fa-4f5730dfb6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-a621d438-40cd-4457-a8fa-4f5730dfb6ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:42.191076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:42.192896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:42.195904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:42.195904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:42.195904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:42.195904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:42.195904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:42.195904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:42.195904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:42.195904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:42.195904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:42.195904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:42.195904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:42.195904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:42.195904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 182/365] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:37:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:42.195904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621d438-40cd-4457-a8fa-4f5730dfb6ec None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:42.293747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e11d9-dcf0-47c3-bff7-36d2d1cb1c2f None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:42.299997 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:42.351258 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621d438-40cd-4457-a8fa-4f5730dfb6ec None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:42.396767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d5f46-7f45-4b52-b097-746364df4662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:42.397682 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d5f46-7f45-4b52-b097-746364df4662 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:42.398153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d5f46-7f45-4b52-b097-746364df4662 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:42.398437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d5f46-7f45-4b52-b097-746364df4662 None admin] PATH_INFO: `/v3/users/5d81d732fa4148edacdb6aaaefa81cdc` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:42.399874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d5f46-7f45-4b52-b097-746364df4662 None admin] RBAC: Authorizing `identity:get_user(user_id=5d81d732fa4148edacdb6aaaefa81cdc)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:42.401399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1d5f46-7f45-4b52-b097-746364df4662 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:42.404954 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 184/366] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:37:42 2022] GET /identity/v3/users/5d81d732fa4148edacdb6aaaefa81cdc => generated 293 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:42.404954 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d5f46-7f45-4b52-b097-746364df4662 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:42.438891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81202ac-f993-462d-8946-83310b34482e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:42.440697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81202ac-f993-462d-8946-83310b34482e None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:42.441727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81202ac-f993-462d-8946-83310b34482e None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:42.442512 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81202ac-f993-462d-8946-83310b34482e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:42.591326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81202ac-f993-462d-8946-83310b34482e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:42.595476 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81202ac-f993-462d-8946-83310b34482e None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:42.615954 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d5f46-7f45-4b52-b097-746364df4662 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:42.635633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:42.679030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abeb16-913c-4dbc-8521-d50ba860a150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:42.680149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abeb16-913c-4dbc-8521-d50ba860a150 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:42.680641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abeb16-913c-4dbc-8521-d50ba860a150 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:42.681120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abeb16-913c-4dbc-8521-d50ba860a150 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:42.777564 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:42.829615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 183/367] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:42.831065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81202ac-f993-462d-8946-83310b34482e None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:42.838720 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abeb16-913c-4dbc-8521-d50ba860a150 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:42.842595 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abeb16-913c-4dbc-8521-d50ba860a150 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:43.003635 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81202ac-f993-462d-8946-83310b34482e None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:43.067574 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 185/368] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 664 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:43.069803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06a082-0064-4827-bddf-6466566ed5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:43.070205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abeb16-913c-4dbc-8521-d50ba860a150 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:43.070981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06a082-0064-4827-bddf-6466566ed5e2 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:43.071496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06a082-0064-4827-bddf-6466566ed5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:43.072037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06a082-0064-4827-bddf-6466566ed5e2 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:43.074531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06a082-0064-4827-bddf-6466566ed5e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:43.075066 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06a082-0064-4827-bddf-6466566ed5e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:43.077324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06a082-0064-4827-bddf-6466566ed5e2 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:43.097090 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 184/369] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:43.099301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06a082-0064-4827-bddf-6466566ed5e2 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:43.114291 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:43.184404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:43.277220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06a082-0064-4827-bddf-6466566ed5e2 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:43.301039 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abeb16-913c-4dbc-8521-d50ba860a150 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:43.337906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047a658-40e3-43ca-8136-cf2b639ec8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:43.340181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047a658-40e3-43ca-8136-cf2b639ec8a2 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:43.341038 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047a658-40e3-43ca-8136-cf2b639ec8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:43.341775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047a658-40e3-43ca-8136-cf2b639ec8a2 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:43.344356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047a658-40e3-43ca-8136-cf2b639ec8a2 None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:43.347475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047a658-40e3-43ca-8136-cf2b639ec8a2 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:43.351557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 185/370] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:43 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:43.353054 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2918c-5d12-4512-bedc-f2109cbfa943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:43.353781 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047a658-40e3-43ca-8136-cf2b639ec8a2 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:43.354172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2918c-5d12-4512-bedc-f2109cbfa943 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:43.354584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2918c-5d12-4512-bedc-f2109cbfa943 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:43.355561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2918c-5d12-4512-bedc-f2109cbfa943 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:43.357183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2918c-5d12-4512-bedc-f2109cbfa943 None admin] RBAC: Adding query filter params (role.id=84bd13e4cee04c12bd6a0e549db79246, user.id=5d81d732fa4148edacdb6aaaefa81cdc, scope.system=all) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:43.357680 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2918c-5d12-4512-bedc-f2109cbfa943 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:43.360307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b2918c-5d12-4512-bedc-f2109cbfa943 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:43.397345 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 186/371] 10.222.0.64 () {56 vars in 1308 bytes} [Thu Jul 14 17:37:43 2022] GET /identity/v3/role_assignments?user.id=5d81d732fa4148edacdb6aaaefa81cdc&scope.system=all&role.id=84bd13e4cee04c12bd6a0e549db79246 => generated 510 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:43.398733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2918c-5d12-4512-bedc-f2109cbfa943 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:43.535461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:43.554336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047a658-40e3-43ca-8136-cf2b639ec8a2 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:43.565520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:43.581618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b2918c-5d12-4512-bedc-f2109cbfa943 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:43.599129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79ea17-bd2b-4db0-ab12-7014495db1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:43.599986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79ea17-bd2b-4db0-ab12-7014495db1b8 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:43.600284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79ea17-bd2b-4db0-ab12-7014495db1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:43.600567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79ea17-bd2b-4db0-ab12-7014495db1b8 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:43.601939 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79ea17-bd2b-4db0-ab12-7014495db1b8 None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:43.603561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79ea17-bd2b-4db0-ab12-7014495db1b8 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:43.611722 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 186/372] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:43 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:43.612815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79ea17-bd2b-4db0-ab12-7014495db1b8 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:43.638507 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb7f6a-11ad-45dd-a9ac-02ad92f46921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:43.639526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb7f6a-11ad-45dd-a9ac-02ad92f46921 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:43.639819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb7f6a-11ad-45dd-a9ac-02ad92f46921 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:43.642498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb7f6a-11ad-45dd-a9ac-02ad92f46921 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:43.720210 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb7f6a-11ad-45dd-a9ac-02ad92f46921 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:43.723043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb7f6a-11ad-45dd-a9ac-02ad92f46921 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:43.745832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79ea17-bd2b-4db0-ab12-7014495db1b8 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:43.788614 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859aafa-97a7-483e-b3f0-8042222501b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:43.789940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859aafa-97a7-483e-b3f0-8042222501b9 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:43.790469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859aafa-97a7-483e-b3f0-8042222501b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:43.790977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859aafa-97a7-483e-b3f0-8042222501b9 None admin] PATH_INFO: `/v3/groups/50fa9d01d91a423bb20869ca7519fea0` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:43.793161 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859aafa-97a7-483e-b3f0-8042222501b9 None admin] RBAC: Authorizing `identity:get_group(group_id=50fa9d01d91a423bb20869ca7519fea0)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:43.795155 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859aafa-97a7-483e-b3f0-8042222501b9 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:43.797328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 187/373] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:43 2022] GET /identity/v3/groups/50fa9d01d91a423bb20869ca7519fea0 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:43.799253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2405880-0d6d-437e-8412-dd2bd41da076 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:43.799812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2405880-0d6d-437e-8412-dd2bd41da076 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:43.800339 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2405880-0d6d-437e-8412-dd2bd41da076 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:43.802373 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 188/374] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:43.803538 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2405880-0d6d-437e-8412-dd2bd41da076 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:43.818479 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:43.849244 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 187/375] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:43.850949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb7f6a-11ad-45dd-a9ac-02ad92f46921 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:43.870351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:43.949088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2405880-0d6d-437e-8412-dd2bd41da076 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:43.981614 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb7f6a-11ad-45dd-a9ac-02ad92f46921 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:43.986027 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bc34cc-3736-4f5c-bd1c-420236c87f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:43.986902 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc34cc-3736-4f5c-bd1c-420236c87f9d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:43.987188 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc34cc-3736-4f5c-bd1c-420236c87f9d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:43.987465 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bc34cc-3736-4f5c-bd1c-420236c87f9d None admin] PATH_INFO: `/v3/groups/50fa9d01d91a423bb20869ca7519fea0` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:43.988681 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc34cc-3736-4f5c-bd1c-420236c87f9d None admin] RBAC: Authorizing `identity:get_group(group_id=50fa9d01d91a423bb20869ca7519fea0)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:43.990141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bc34cc-3736-4f5c-bd1c-420236c87f9d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:43.997619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 189/376] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:43 2022] GET /identity/v3/groups/50fa9d01d91a423bb20869ca7519fea0 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:43.997619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf780d3-8dfc-4dbf-a4d6-bb07fc6bfab8 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:43.997619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf780d3-8dfc-4dbf-a4d6-bb07fc6bfab8 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:43.997619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf780d3-8dfc-4dbf-a4d6-bb07fc6bfab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:44.013848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-daf780d3-8dfc-4dbf-a4d6-bb07fc6bfab8 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:44.015130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-daf780d3-8dfc-4dbf-a4d6-bb07fc6bfab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:44.025234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a79918-42de-4ec5-9000-dde1017049ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:44.025941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a79918-42de-4ec5-9000-dde1017049ab None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:44.026254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a79918-42de-4ec5-9000-dde1017049ab None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:44.026539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a79918-42de-4ec5-9000-dde1017049ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:44.027547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a79918-42de-4ec5-9000-dde1017049ab None admin] RBAC: Adding query filter params (group.id=50fa9d01d91a423bb20869ca7519fea0, role.id=8427479f632a4da3949971f58b438c7f, scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:44.027765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a79918-42de-4ec5-9000-dde1017049ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:44.029255 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a79918-42de-4ec5-9000-dde1017049ab None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:44.035245 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 188/377] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Jul 14 17:37:43 2022] GET /identity/v3/role_assignments?group.id=50fa9d01d91a423bb20869ca7519fea0&scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82&role.id=8427479f632a4da3949971f58b438c7f => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:44.035868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a79918-42de-4ec5-9000-dde1017049ab None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:44.088195 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:44.155486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 190/378] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:44.156923 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf780d3-8dfc-4dbf-a4d6-bb07fc6bfab8 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:44.172169 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a79918-42de-4ec5-9000-dde1017049ab None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:44.216341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:44.224930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a14bb9-ed2b-4970-a7f5-948b158fa888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:44.225821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a14bb9-ed2b-4970-a7f5-948b158fa888 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:44.226105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a14bb9-ed2b-4970-a7f5-948b158fa888 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:44.226391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a14bb9-ed2b-4970-a7f5-948b158fa888 None admin] PATH_INFO: `/v3/projects/29b8bf26bc324367a4fe27e90fc2d6df` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:44.228634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a14bb9-ed2b-4970-a7f5-948b158fa888 None admin] RBAC: Authorizing `identity:get_project(project_id=29b8bf26bc324367a4fe27e90fc2d6df)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:44.230173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a14bb9-ed2b-4970-a7f5-948b158fa888 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:44.232617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 189/379] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:44 2022] GET /identity/v3/projects/29b8bf26bc324367a4fe27e90fc2d6df => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:44.232815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a14bb9-ed2b-4970-a7f5-948b158fa888 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:44.298680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf780d3-8dfc-4dbf-a4d6-bb07fc6bfab8 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:44.350998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85ec29-a0e7-4cd1-9fa6-bc6a3ec46008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:44.352699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85ec29-a0e7-4cd1-9fa6-bc6a3ec46008 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:44.353523 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85ec29-a0e7-4cd1-9fa6-bc6a3ec46008 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:44.354288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85ec29-a0e7-4cd1-9fa6-bc6a3ec46008 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:44.356994 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85ec29-a0e7-4cd1-9fa6-bc6a3ec46008 None admin] RBAC: Adding query filter params (group.id=50fa9d01d91a423bb20869ca7519fea0, role.id=8427479f632a4da3949971f58b438c7f, scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:44.357897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85ec29-a0e7-4cd1-9fa6-bc6a3ec46008 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:44.361101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf85ec29-a0e7-4cd1-9fa6-bc6a3ec46008 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:44.362535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a14bb9-ed2b-4970-a7f5-948b158fa888 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:44.371627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:44.380056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 191/380] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Jul 14 17:37:44 2022] GET /identity/v3/role_assignments?group.id=50fa9d01d91a423bb20869ca7519fea0&scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2&role.id=8427479f632a4da3949971f58b438c7f => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:44.382169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85ec29-a0e7-4cd1-9fa6-bc6a3ec46008 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:44.405965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a876e1-e375-4d96-9b03-f5aff711ea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:44.406780 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a876e1-e375-4d96-9b03-f5aff711ea2e None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:44.407125 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a876e1-e375-4d96-9b03-f5aff711ea2e None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:44.407409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a876e1-e375-4d96-9b03-f5aff711ea2e None admin] PATH_INFO: `/v3/roles/member` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:44.425707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a876e1-e375-4d96-9b03-f5aff711ea2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-77a876e1-e375-4d96-9b03-f5aff711ea2e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:44.435284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:44.437579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:44.439776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:44.439776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:44.439776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:44.439776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:44.439776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:44.439776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:44.439776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:44.439776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:44.439776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:44.439776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:44.439776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:44.439776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:44.439776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 190/381] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:44.445792 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a876e1-e375-4d96-9b03-f5aff711ea2e None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:44.601332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85ec29-a0e7-4cd1-9fa6-bc6a3ec46008 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:44.619250 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:44.684209 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a876e1-e375-4d96-9b03-f5aff711ea2e None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:44.704231 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3edb39-0cb2-4891-9054-82a76212897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:44.705504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3edb39-0cb2-4891-9054-82a76212897a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:44.705993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3edb39-0cb2-4891-9054-82a76212897a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:44.706478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3edb39-0cb2-4891-9054-82a76212897a None admin] PATH_INFO: `/v3/groups/a50abba7e46e484a8d50a3f54750b8f3` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:44.731026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3edb39-0cb2-4891-9054-82a76212897a None admin] RBAC: Authorizing `identity:get_group(group_id=a50abba7e46e484a8d50a3f54750b8f3)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:44.733205 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3edb39-0cb2-4891-9054-82a76212897a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:44.735076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 192/382] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:44 2022] GET /identity/v3/groups/a50abba7e46e484a8d50a3f54750b8f3 => generated 229 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:44.735846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec90e03-21a9-4917-bb62-a3c3bf1258ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:44.736661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec90e03-21a9-4917-bb62-a3c3bf1258ab None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:44.736967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec90e03-21a9-4917-bb62-a3c3bf1258ab None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:44.737248 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec90e03-21a9-4917-bb62-a3c3bf1258ab None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:44.738046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec90e03-21a9-4917-bb62-a3c3bf1258ab None admin] RBAC: Adding query filter params (name=member) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:44.738359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec90e03-21a9-4917-bb62-a3c3bf1258ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:44.739784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec90e03-21a9-4917-bb62-a3c3bf1258ab None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:44.741318 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3edb39-0cb2-4891-9054-82a76212897a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:44.760976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 191/383] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:44.767943 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec90e03-21a9-4917-bb62-a3c3bf1258ab None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:44.800581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:44.899877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec90e03-21a9-4917-bb62-a3c3bf1258ab None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:44.942150 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c25c9-d785-41ae-8f4f-c2ca9f5805ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:44.943440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c25c9-d785-41ae-8f4f-c2ca9f5805ef None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:44.943958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c25c9-d785-41ae-8f4f-c2ca9f5805ef None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:44.944499 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9c25c9-d785-41ae-8f4f-c2ca9f5805ef None admin] PATH_INFO: `/v3/groups/50fa9d01d91a423bb20869ca7519fea0` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:44.946118 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c25c9-d785-41ae-8f4f-c2ca9f5805ef None admin] RBAC: Authorizing `identity:get_group(group_id=50fa9d01d91a423bb20869ca7519fea0)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:44.947823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9c25c9-d785-41ae-8f4f-c2ca9f5805ef None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:44.950650 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 192/384] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:44 2022] GET /identity/v3/groups/50fa9d01d91a423bb20869ca7519fea0 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:44.960099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c25c9-d785-41ae-8f4f-c2ca9f5805ef None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:44.988398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3edb39-0cb2-4891-9054-82a76212897a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:45.039128 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9ea50-86d5-41af-9e2f-82ccf72361c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:45.040308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9ea50-86d5-41af-9e2f-82ccf72361c3 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:45.040814 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9ea50-86d5-41af-9e2f-82ccf72361c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:45.041317 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9ea50-86d5-41af-9e2f-82ccf72361c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:45.043006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9ea50-86d5-41af-9e2f-82ccf72361c3 None admin] RBAC: Adding query filter params (group.id=a50abba7e46e484a8d50a3f54750b8f3, role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=29b8bf26bc324367a4fe27e90fc2d6df) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:45.043579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9ea50-86d5-41af-9e2f-82ccf72361c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:45.048777 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a9ea50-86d5-41af-9e2f-82ccf72361c3 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:45.071958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 193/385] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Jul 14 17:37:44 2022] GET /identity/v3/role_assignments?group.id=a50abba7e46e484a8d50a3f54750b8f3&scope.project.id=29b8bf26bc324367a4fe27e90fc2d6df&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 260 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:45.204477 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:45.218902 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badffb8-fbfd-4298-8fa2-e47145afaf19 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:45.219413 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badffb8-fbfd-4298-8fa2-e47145afaf19 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:45.219942 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badffb8-fbfd-4298-8fa2-e47145afaf19 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:45.222643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9c25c9-d785-41ae-8f4f-c2ca9f5805ef None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:45.229394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 194/386] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:45 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:45.240507 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75d452-0175-4dd1-91c1-5dd6f6faab2b None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:45.240916 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75d452-0175-4dd1-91c1-5dd6f6faab2b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:45.241109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75d452-0175-4dd1-91c1-5dd6f6faab2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:45.278190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-1b75d452-0175-4dd1-91c1-5dd6f6faab2b None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:45.279762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1b75d452-0175-4dd1-91c1-5dd6f6faab2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:45.307189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40113ed7-27a1-4c7e-8dc1-d86e573bdfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:45.309042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40113ed7-27a1-4c7e-8dc1-d86e573bdfc2 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:45.309534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40113ed7-27a1-4c7e-8dc1-d86e573bdfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:45.310009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40113ed7-27a1-4c7e-8dc1-d86e573bdfc2 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:45.311505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40113ed7-27a1-4c7e-8dc1-d86e573bdfc2 None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:45.313265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40113ed7-27a1-4c7e-8dc1-d86e573bdfc2 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:45.315535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 193/387] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:44 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:45.329408 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40113ed7-27a1-4c7e-8dc1-d86e573bdfc2 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:45.338439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:45.426442 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 195/388] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:45.463402 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75d452-0175-4dd1-91c1-5dd6f6faab2b None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:45.532681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40113ed7-27a1-4c7e-8dc1-d86e573bdfc2 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:45.594709 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c7d26-1fd9-4c15-a4bf-a486bb7a4952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:45.596312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c7d26-1fd9-4c15-a4bf-a486bb7a4952 None admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:45.597645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c7d26-1fd9-4c15-a4bf-a486bb7a4952 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:45.598134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c7d26-1fd9-4c15-a4bf-a486bb7a4952 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82/groups/50fa9d01d91a423bb20869ca7519fea0/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:45.604935 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c7d26-1fd9-4c15-a4bf-a486bb7a4952 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba02b19bd80d44c4a9a9ae6517b11b82, group_id=50fa9d01d91a423bb20869ca7519fea0, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:45.607260 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399c7d26-1fd9-4c15-a4bf-a486bb7a4952 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:45.630321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 194/389] 10.222.0.64 () {58 vars in 1442 bytes} [Thu Jul 14 17:37:45 2022] PUT /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82/groups/50fa9d01d91a423bb20869ca7519fea0/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:37:45.647405 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75d452-0175-4dd1-91c1-5dd6f6faab2b None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:45.690132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8b80c-7176-4dca-abff-d94fcec49764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:45.691458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8b80c-7176-4dca-abff-d94fcec49764 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:45.692027 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8b80c-7176-4dca-abff-d94fcec49764 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:45.692507 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac8b80c-7176-4dca-abff-d94fcec49764 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:45.712110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac8b80c-7176-4dca-abff-d94fcec49764 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-aac8b80c-7176-4dca-abff-d94fcec49764 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:45.721464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:45.722446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:45.723234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:45.723234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:45.723234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:45.723234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:45.723234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:45.723234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:45.723234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:45.723234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:45.723234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:45.723234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:45.723234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:45.723234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:45.725868 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 196/390] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:45.733459 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c7d26-1fd9-4c15-a4bf-a486bb7a4952 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:45.815160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:45.931376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c7d26-1fd9-4c15-a4bf-a486bb7a4952 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:45.972518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1cddb7-d2aa-44ca-9202-d90f9eb97039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:45.973660 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cddb7-d2aa-44ca-9202-d90f9eb97039 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:45.974152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cddb7-d2aa-44ca-9202-d90f9eb97039 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:45.974644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cddb7-d2aa-44ca-9202-d90f9eb97039 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:45.975615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1cddb7-d2aa-44ca-9202-d90f9eb97039 None admin] RBAC: Adding query filter params (name=member) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:45.976159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1cddb7-d2aa-44ca-9202-d90f9eb97039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:45.977809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1cddb7-d2aa-44ca-9202-d90f9eb97039 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:45.989903 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 195/391] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:46.003164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8b80c-7176-4dca-abff-d94fcec49764 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:46.054954 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:46.171685 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac8b80c-7176-4dca-abff-d94fcec49764 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:46.225966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7383e0-71ad-4edd-b936-36c144475e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:46.226771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7383e0-71ad-4edd-b936-36c144475e95 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:46.227054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7383e0-71ad-4edd-b936-36c144475e95 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:46.227330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7383e0-71ad-4edd-b936-36c144475e95 None admin] PATH_INFO: `/v3/groups/50fa9d01d91a423bb20869ca7519fea0` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:46.229583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7383e0-71ad-4edd-b936-36c144475e95 None admin] RBAC: Authorizing `identity:get_group(group_id=50fa9d01d91a423bb20869ca7519fea0)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:46.231613 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7383e0-71ad-4edd-b936-36c144475e95 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:46.233808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 197/392] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:46 2022] GET /identity/v3/groups/50fa9d01d91a423bb20869ca7519fea0 => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:46.241634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1cddb7-d2aa-44ca-9202-d90f9eb97039 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:46.420345 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:46.448187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1cddb7-d2aa-44ca-9202-d90f9eb97039 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:46.501621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8bea8-aaf9-4845-99dc-21d8188fdcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:46.502427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8bea8-aaf9-4845-99dc-21d8188fdcbf None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:46.502714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8bea8-aaf9-4845-99dc-21d8188fdcbf None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:46.502997 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8bea8-aaf9-4845-99dc-21d8188fdcbf None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:46.505875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8bea8-aaf9-4845-99dc-21d8188fdcbf None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:46.507493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8bea8-aaf9-4845-99dc-21d8188fdcbf None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:46.509751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 196/393] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:46 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:46.515196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7383e0-71ad-4edd-b936-36c144475e95 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:46.658146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:46.667382 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7383e0-71ad-4edd-b936-36c144475e95 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:46.679017 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61021843-e26c-4e3c-9ac1-1fd1775356ad None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:46.679335 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61021843-e26c-4e3c-9ac1-1fd1775356ad None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:46.679595 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61021843-e26c-4e3c-9ac1-1fd1775356ad None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:46.681160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 197/394] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:37:46.689400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d48fa-efd2-4bc3-822c-d94b3158593d None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:46.689708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d48fa-efd2-4bc3-822c-d94b3158593d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:46.689971 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d48fa-efd2-4bc3-822c-d94b3158593d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:46.717812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-ac2d48fa-efd2-4bc3-822c-d94b3158593d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:46.720764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ac2d48fa-efd2-4bc3-822c-d94b3158593d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:46.726346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aee496-5559-4f2d-bc07-c0cad29ed827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:46.727637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aee496-5559-4f2d-bc07-c0cad29ed827 None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:46.728128 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aee496-5559-4f2d-bc07-c0cad29ed827 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:46.728625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aee496-5559-4f2d-bc07-c0cad29ed827 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2/groups/50fa9d01d91a423bb20869ca7519fea0/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:46.731833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aee496-5559-4f2d-bc07-c0cad29ed827 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bebbd67fc624e7ea047e2d52573e5b2, group_id=50fa9d01d91a423bb20869ca7519fea0, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:46.734407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aee496-5559-4f2d-bc07-c0cad29ed827 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:46.762877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 198/395] 10.222.0.64 () {58 vars in 1442 bytes} [Thu Jul 14 17:37:46 2022] PUT /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2/groups/50fa9d01d91a423bb20869ca7519fea0/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:37:46.887579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 198/396] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:46.927651 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aee496-5559-4f2d-bc07-c0cad29ed827 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:47.029437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7307a-45de-4282-86e4-28937b36b85f None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:47.029694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7307a-45de-4282-86e4-28937b36b85f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:47.030091 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7307a-45de-4282-86e4-28937b36b85f None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:47.031880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 199/397] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:47.037216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f9ad6-21ed-4446-aa61-196f8c83a400 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:47.037579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f9ad6-21ed-4446-aa61-196f8c83a400 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:47.037946 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f9ad6-21ed-4446-aa61-196f8c83a400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:47.056376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aee496-5559-4f2d-bc07-c0cad29ed827 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:47.057195 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-911f9ad6-21ed-4446-aa61-196f8c83a400 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:47.059132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-911f9ad6-21ed-4446-aa61-196f8c83a400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:47.096265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58134b90-ecb6-4066-95fa-96b553298c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:47.097431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58134b90-ecb6-4066-95fa-96b553298c2c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:47.097921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58134b90-ecb6-4066-95fa-96b553298c2c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:47.098368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58134b90-ecb6-4066-95fa-96b553298c2c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:47.111180 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58134b90-ecb6-4066-95fa-96b553298c2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-58134b90-ecb6-4066-95fa-96b553298c2c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:47.121740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:47.122636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:47.123427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:47.123427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:47.123427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:47.123427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:47.123427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:47.123427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:47.123427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:47.123427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:47.123427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:47.123427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:47.123427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:47.123427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:47.131094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 199/398] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:37:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:47.132720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58134b90-ecb6-4066-95fa-96b553298c2c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:47.145028 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 200/399] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:47.160332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:47.168338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f9ad6-21ed-4446-aa61-196f8c83a400 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:47.184342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:47.253837 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58134b90-ecb6-4066-95fa-96b553298c2c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:47.278167 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f9ad6-21ed-4446-aa61-196f8c83a400 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:47.292908 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2449f5f-902d-4690-be96-aae9adbb177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:47.293983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2449f5f-902d-4690-be96-aae9adbb177f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:47.294467 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2449f5f-902d-4690-be96-aae9adbb177f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:47.294962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2449f5f-902d-4690-be96-aae9adbb177f None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:47.296054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2449f5f-902d-4690-be96-aae9adbb177f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:47.296657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2449f5f-902d-4690-be96-aae9adbb177f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:47.298270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2449f5f-902d-4690-be96-aae9adbb177f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:47.313352 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 200/400] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:37:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:47.315301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2449f5f-902d-4690-be96-aae9adbb177f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:47.316818 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5cf41-9496-43fc-8466-e9a32106465b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:47.317677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5cf41-9496-43fc-8466-e9a32106465b None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:47.317965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5cf41-9496-43fc-8466-e9a32106465b None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:47.318246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d5cf41-9496-43fc-8466-e9a32106465b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:47.331357 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d5cf41-9496-43fc-8466-e9a32106465b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-b6d5cf41-9496-43fc-8466-e9a32106465b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:47.339742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:47.340745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:47.341953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:47.341953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:47.341953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:47.341953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:47.341953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:47.341953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:47.341953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:47.341953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:47.341953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:47.341953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:47.341953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:47.341953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:47.341953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 201/401] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:37:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:47.347499 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5cf41-9496-43fc-8466-e9a32106465b None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:47.403052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:47.429968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2449f5f-902d-4690-be96-aae9adbb177f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:47.467667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7c006-781f-413e-bb23-544e5ae2b5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:47.468944 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7c006-781f-413e-bb23-544e5ae2b5d5 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:47.469428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7c006-781f-413e-bb23-544e5ae2b5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:47.469934 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c7c006-781f-413e-bb23-544e5ae2b5d5 None admin] PATH_INFO: `/v3/groups/50fa9d01d91a423bb20869ca7519fea0` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:47.471320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7c006-781f-413e-bb23-544e5ae2b5d5 None admin] RBAC: Authorizing `identity:get_group(group_id=50fa9d01d91a423bb20869ca7519fea0)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:47.473183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:47.474262 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c7c006-781f-413e-bb23-544e5ae2b5d5 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:47.480132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 201/402] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:47 2022] GET /identity/v3/groups/50fa9d01d91a423bb20869ca7519fea0 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:47.483870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7c006-781f-413e-bb23-544e5ae2b5d5 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:47.521730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d5cf41-9496-43fc-8466-e9a32106465b None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:47.594826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c7c006-781f-413e-bb23-544e5ae2b5d5 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:47.614101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4259a0-a5eb-4581-994b-ee31e3a2db86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:47.615280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4259a0-a5eb-4581-994b-ee31e3a2db86 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:47.615793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4259a0-a5eb-4581-994b-ee31e3a2db86 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:47.616332 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4259a0-a5eb-4581-994b-ee31e3a2db86 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:47.617804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4259a0-a5eb-4581-994b-ee31e3a2db86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:47.618491 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4259a0-a5eb-4581-994b-ee31e3a2db86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:47.621921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4259a0-a5eb-4581-994b-ee31e3a2db86 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:47.633874 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b776a5-db12-4315-9365-b31c597fbbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:47.635082 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b776a5-db12-4315-9365-b31c597fbbe6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:47.635567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b776a5-db12-4315-9365-b31c597fbbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:47.636116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b776a5-db12-4315-9365-b31c597fbbe6 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:47.637681 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b776a5-db12-4315-9365-b31c597fbbe6 None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:47.639471 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b776a5-db12-4315-9365-b31c597fbbe6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:47.641435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 202/403] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:47 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:47.647324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b776a5-db12-4315-9365-b31c597fbbe6 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:47.659195 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 202/404] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:37:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:47.662140 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:47.668973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4259a0-a5eb-4581-994b-ee31e3a2db86 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:47.758813 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b776a5-db12-4315-9365-b31c597fbbe6 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:47.795548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62aeec-6b39-4b65-92fe-ea9afd691ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:47.796839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62aeec-6b39-4b65-92fe-ea9afd691ed4 None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:47.797327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62aeec-6b39-4b65-92fe-ea9afd691ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:47.797801 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62aeec-6b39-4b65-92fe-ea9afd691ed4 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82/groups/50fa9d01d91a423bb20869ca7519fea0/roles/8427479f632a4da3949971f58b438c7f` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:47.800370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62aeec-6b39-4b65-92fe-ea9afd691ed4 None admin] RBAC: Authorizing `identity:create_grant(project_id=ba02b19bd80d44c4a9a9ae6517b11b82, group_id=50fa9d01d91a423bb20869ca7519fea0, role_id=8427479f632a4da3949971f58b438c7f)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:47.802072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62aeec-6b39-4b65-92fe-ea9afd691ed4 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:47.840992 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 203/405] 10.222.0.64 () {58 vars in 1442 bytes} [Thu Jul 14 17:37:47 2022] PUT /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82/groups/50fa9d01d91a423bb20869ca7519fea0/roles/8427479f632a4da3949971f58b438c7f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:37:47.928543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4259a0-a5eb-4581-994b-ee31e3a2db86 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:47.997728 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d16c1-550f-4330-b24f-f2187ec95d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:47.999216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d16c1-550f-4330-b24f-f2187ec95d83 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:47.999717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d16c1-550f-4330-b24f-f2187ec95d83 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:48.000242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d16c1-550f-4330-b24f-f2187ec95d83 None admin] PATH_INFO: `/v3/groups/50fa9d01d91a423bb20869ca7519fea0` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:48.002443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d16c1-550f-4330-b24f-f2187ec95d83 None admin] RBAC: Authorizing `identity:get_group(group_id=50fa9d01d91a423bb20869ca7519fea0)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:48.005201 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d16c1-550f-4330-b24f-f2187ec95d83 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:48.008344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 203/406] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:47 2022] GET /identity/v3/groups/50fa9d01d91a423bb20869ca7519fea0 => generated 226 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:48.017587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62aeec-6b39-4b65-92fe-ea9afd691ed4 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:48.127455 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:48.168658 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62aeec-6b39-4b65-92fe-ea9afd691ed4 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:48.216746 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec44313-596d-49d4-b1f5-4ebc74bb83c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:48.217674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec44313-596d-49d4-b1f5-4ebc74bb83c9 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:48.217960 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec44313-596d-49d4-b1f5-4ebc74bb83c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:48.218236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec44313-596d-49d4-b1f5-4ebc74bb83c9 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:48.219684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec44313-596d-49d4-b1f5-4ebc74bb83c9 None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:48.221251 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec44313-596d-49d4-b1f5-4ebc74bb83c9 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:48.223296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 204/407] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:48 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:48.224669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d1f73-ccdc-42fa-aee3-75539768eb2c None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:48.225214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d1f73-ccdc-42fa-aee3-75539768eb2c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:48.225694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d1f73-ccdc-42fa-aee3-75539768eb2c None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:48.227515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 204/408] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:48.229971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec44313-596d-49d4-b1f5-4ebc74bb83c9 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:48.233065 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fa91f-9692-4bb8-b799-f7c8e2249d30 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:48.233603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fa91f-9692-4bb8-b799-f7c8e2249d30 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:48.234100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fa91f-9692-4bb8-b799-f7c8e2249d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:48.241172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:48.262881 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-d62fa91f-9692-4bb8-b799-f7c8e2249d30 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:48.264476 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d62fa91f-9692-4bb8-b799-f7c8e2249d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:48.353512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 205/409] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:48.376747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62fa91f-9692-4bb8-b799-f7c8e2249d30 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:48.422351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec44313-596d-49d4-b1f5-4ebc74bb83c9 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:48.461654 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cd97f-21bf-441b-a2d1-19a577779142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:48.463015 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cd97f-21bf-441b-a2d1-19a577779142 None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:48.463660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cd97f-21bf-441b-a2d1-19a577779142 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:48.464172 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cd97f-21bf-441b-a2d1-19a577779142 None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2/groups/50fa9d01d91a423bb20869ca7519fea0/roles/8427479f632a4da3949971f58b438c7f` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:48.467382 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cd97f-21bf-441b-a2d1-19a577779142 None admin] RBAC: Authorizing `identity:create_grant(project_id=1bebbd67fc624e7ea047e2d52573e5b2, group_id=50fa9d01d91a423bb20869ca7519fea0, role_id=8427479f632a4da3949971f58b438c7f)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:48.469192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6cd97f-21bf-441b-a2d1-19a577779142 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:48.479501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:48.486643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 205/410] 10.222.0.64 () {58 vars in 1442 bytes} [Thu Jul 14 17:37:48 2022] PUT /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2/groups/50fa9d01d91a423bb20869ca7519fea0/roles/8427479f632a4da3949971f58b438c7f => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:37:48.488971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1cf462-83cd-4261-a841-fdffcef53482 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:48.489634 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1cf462-83cd-4261-a841-fdffcef53482 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:48.490137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1cf462-83cd-4261-a841-fdffcef53482 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:48.492232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 206/411] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:48.504368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaac84-dae7-4b38-8882-ab6dbb7e789e None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:48.505571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaac84-dae7-4b38-8882-ab6dbb7e789e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:48.506349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfaac84-dae7-4b38-8882-ab6dbb7e789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:48.515968 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62fa91f-9692-4bb8-b799-f7c8e2249d30 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:48.532113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-6bfaac84-dae7-4b38-8882-ab6dbb7e789e None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:48.533684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6bfaac84-dae7-4b38-8882-ab6dbb7e789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:48.587340 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ef4bf-e8e0-445c-a6ab-18e900438de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:48.588377 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ef4bf-e8e0-445c-a6ab-18e900438de1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:48.588761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ef4bf-e8e0-445c-a6ab-18e900438de1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:48.589113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ef4bf-e8e0-445c-a6ab-18e900438de1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:48.595694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:48.628025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ef4bf-e8e0-445c-a6ab-18e900438de1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:48.631020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 207/412] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-5e2ef4bf-e8e0-445c-a6ab-18e900438de1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:48.636421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:48.637429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:48.638230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:48.638230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:48.638230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:48.638230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:48.638230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:48.638230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:48.638230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:48.638230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:48.638230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:48.638230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:48.638230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:48.638230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:48.642686 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 206/413] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:37:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:48.644093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaac84-dae7-4b38-8882-ab6dbb7e789e None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:48.662858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ef4bf-e8e0-445c-a6ab-18e900438de1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:48.825852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfaac84-dae7-4b38-8882-ab6dbb7e789e None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:48.835383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ef4bf-e8e0-445c-a6ab-18e900438de1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:48.858688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:48.870416 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc282f3-e725-4b9e-96d9-fb4f03d49f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:48.871844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febd18b-83a2-4ead-8403-7dc81af29cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:48.872198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc282f3-e725-4b9e-96d9-fb4f03d49f3b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:48.872639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc282f3-e725-4b9e-96d9-fb4f03d49f3b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:48.872888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febd18b-83a2-4ead-8403-7dc81af29cbb None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:48.873211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc282f3-e725-4b9e-96d9-fb4f03d49f3b None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:48.873320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febd18b-83a2-4ead-8403-7dc81af29cbb None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:48.873622 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febd18b-83a2-4ead-8403-7dc81af29cbb None admin] PATH_INFO: `/v3/roles/member` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:48.874650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc282f3-e725-4b9e-96d9-fb4f03d49f3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:48.875108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc282f3-e725-4b9e-96d9-fb4f03d49f3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:48.877529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc282f3-e725-4b9e-96d9-fb4f03d49f3b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:48.888598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 208/414] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:48.896388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc282f3-e725-4b9e-96d9-fb4f03d49f3b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:48.901706 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febd18b-83a2-4ead-8403-7dc81af29cbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-8febd18b-83a2-4ead-8403-7dc81af29cbb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:48.910600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:48.911557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:48.912376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:48.912376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:48.912376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:48.912376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:48.912376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:48.912376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:48.912376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:48.912376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:48.912376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:48.912376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:48.912376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:48.912376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:48.912376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 207/415] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:48.922982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febd18b-83a2-4ead-8403-7dc81af29cbb None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:49.031465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:49.086880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc282f3-e725-4b9e-96d9-fb4f03d49f3b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:49.091301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febd18b-83a2-4ead-8403-7dc81af29cbb None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:49.149879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b665a-d2fa-4df4-8003-8a60bcc842fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:49.151037 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b665a-d2fa-4df4-8003-8a60bcc842fb None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:49.155536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b665a-d2fa-4df4-8003-8a60bcc842fb None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:49.156182 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0b665a-d2fa-4df4-8003-8a60bcc842fb None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:49.157200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b665a-d2fa-4df4-8003-8a60bcc842fb None admin] RBAC: Adding query filter params (name=member) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:49.157717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b665a-d2fa-4df4-8003-8a60bcc842fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:49.159373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0b665a-d2fa-4df4-8003-8a60bcc842fb None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:49.179814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 208/416] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:49.186405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b665a-d2fa-4df4-8003-8a60bcc842fb None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:49.191484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8528120-4fd6-4984-af97-eb1760fd177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:49.192403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8528120-4fd6-4984-af97-eb1760fd177e None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:49.192691 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8528120-4fd6-4984-af97-eb1760fd177e None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:49.192970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8528120-4fd6-4984-af97-eb1760fd177e None admin] PATH_INFO: `/v3/groups/a50abba7e46e484a8d50a3f54750b8f3` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:49.195350 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8528120-4fd6-4984-af97-eb1760fd177e None admin] RBAC: Authorizing `identity:get_group(group_id=a50abba7e46e484a8d50a3f54750b8f3)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:49.196694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8528120-4fd6-4984-af97-eb1760fd177e None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:49.204940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 209/417] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:48 2022] GET /identity/v3/groups/a50abba7e46e484a8d50a3f54750b8f3 => generated 229 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:49.208143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8528120-4fd6-4984-af97-eb1760fd177e None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:49.338632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:49.400845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0b665a-d2fa-4df4-8003-8a60bcc842fb None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:49.420836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8528120-4fd6-4984-af97-eb1760fd177e None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:49.442159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:49.484070 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21d15f-af94-499f-a0ce-fb213686577f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:49.485580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21d15f-af94-499f-a0ce-fb213686577f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:49.486075 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21d15f-af94-499f-a0ce-fb213686577f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:49.486562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21d15f-af94-499f-a0ce-fb213686577f None admin] PATH_INFO: `/v3/projects/29b8bf26bc324367a4fe27e90fc2d6df` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:49.491331 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21d15f-af94-499f-a0ce-fb213686577f None admin] RBAC: Authorizing `identity:get_project(project_id=29b8bf26bc324367a4fe27e90fc2d6df)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:49.494231 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21d15f-af94-499f-a0ce-fb213686577f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:49.507750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 210/418] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:49 2022] GET /identity/v3/projects/29b8bf26bc324367a4fe27e90fc2d6df => generated 344 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:49.508988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0b9ae-5639-44ff-b807-eb914d4f07ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:49.510058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0b9ae-5639-44ff-b807-eb914d4f07ba None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:49.510601 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0b9ae-5639-44ff-b807-eb914d4f07ba None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:49.511084 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd0b9ae-5639-44ff-b807-eb914d4f07ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:49.513818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21d15f-af94-499f-a0ce-fb213686577f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:49.688629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd0b9ae-5639-44ff-b807-eb914d4f07ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:49.690263 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd0b9ae-5639-44ff-b807-eb914d4f07ba None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:49.728574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21d15f-af94-499f-a0ce-fb213686577f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:49.758936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:49.771726 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7d488-c90b-43aa-bc46-b03c711ea6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:49.772845 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7d488-c90b-43aa-bc46-b03c711ea6b0 None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:49.773069 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7d488-c90b-43aa-bc46-b03c711ea6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:49.773353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7d488-c90b-43aa-bc46-b03c711ea6b0 None admin] PATH_INFO: `/v3/projects/29b8bf26bc324367a4fe27e90fc2d6df/groups/a50abba7e46e484a8d50a3f54750b8f3/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:49.775444 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7d488-c90b-43aa-bc46-b03c711ea6b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=29b8bf26bc324367a4fe27e90fc2d6df, group_id=a50abba7e46e484a8d50a3f54750b8f3, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:49.776994 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7d488-c90b-43aa-bc46-b03c711ea6b0 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:49.796194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 211/419] 10.222.0.64 () {58 vars in 1442 bytes} [Thu Jul 14 17:37:49 2022] PUT /identity/v3/projects/29b8bf26bc324367a4fe27e90fc2d6df/groups/a50abba7e46e484a8d50a3f54750b8f3/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:37:49.797823 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aea60f-b06d-4c98-a717-e01a07e92fa7 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:49.798121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aea60f-b06d-4c98-a717-e01a07e92fa7 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:49.798403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aea60f-b06d-4c98-a717-e01a07e92fa7 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:49.804300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 212/420] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:49.805818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4e708-ecff-47b6-ac5f-2b0f2b4de82a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:49.806168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4e708-ecff-47b6-ac5f-2b0f2b4de82a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:49.806461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4e708-ecff-47b6-ac5f-2b0f2b4de82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:49.830798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-dbe4e708-ecff-47b6-ac5f-2b0f2b4de82a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:49.832070 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-dbe4e708-ecff-47b6-ac5f-2b0f2b4de82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:49.944684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 209/421] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 759 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:49.953526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0b9ae-5639-44ff-b807-eb914d4f07ba None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:49.960088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 213/422] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:49.986193 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4e708-ecff-47b6-ac5f-2b0f2b4de82a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:49.987804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:50.153272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd0b9ae-5639-44ff-b807-eb914d4f07ba None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:50.215748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4e708-ecff-47b6-ac5f-2b0f2b4de82a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:50.217399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf772f-32a4-4be4-bf0f-6da2a9e0df37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:50.218274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf772f-32a4-4be4-bf0f-6da2a9e0df37 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:50.218554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf772f-32a4-4be4-bf0f-6da2a9e0df37 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:50.218826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf772f-32a4-4be4-bf0f-6da2a9e0df37 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:50.223957 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf772f-32a4-4be4-bf0f-6da2a9e0df37 None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:50.223957 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf772f-32a4-4be4-bf0f-6da2a9e0df37 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:50.223957 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 210/423] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:49 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:50.234062 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf772f-32a4-4be4-bf0f-6da2a9e0df37 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:50.262949 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:50.306562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf99d3-970f-437b-b021-0ec78e54f631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:50.307449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf99d3-970f-437b-b021-0ec78e54f631 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:50.307727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf99d3-970f-437b-b021-0ec78e54f631 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:50.308029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf99d3-970f-437b-b021-0ec78e54f631 None admin] PATH_INFO: `/v3/roles/member` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:50.330032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf99d3-970f-437b-b021-0ec78e54f631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-9bbf99d3-970f-437b-b021-0ec78e54f631 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:50.346341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:50.348190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:50.349782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:50.349782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:50.349782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:50.349782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:50.349782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:50.349782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:50.349782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:50.349782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:50.349782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:50.349782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:50.349782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 17:37:50.349782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:50.349782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 214/424] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:50.360148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf99d3-970f-437b-b021-0ec78e54f631 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:50.387251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:50.409908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf772f-32a4-4be4-bf0f-6da2a9e0df37 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:50.460403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79c31f-38ab-40f9-8765-fb75f5f77953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:50.461511 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79c31f-38ab-40f9-8765-fb75f5f77953 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:50.461898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79c31f-38ab-40f9-8765-fb75f5f77953 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:50.462535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79c31f-38ab-40f9-8765-fb75f5f77953 None admin] PATH_INFO: `/v3/groups/50fa9d01d91a423bb20869ca7519fea0` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:50.464414 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79c31f-38ab-40f9-8765-fb75f5f77953 None admin] RBAC: Authorizing `identity:get_group(group_id=50fa9d01d91a423bb20869ca7519fea0)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:50.467015 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79c31f-38ab-40f9-8765-fb75f5f77953 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:50.469038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 211/425] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:50 2022] GET /identity/v3/groups/50fa9d01d91a423bb20869ca7519fea0 => generated 226 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:50.473465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1437a-0587-4228-b0ae-304abbc2456f None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:50.473784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1437a-0587-4228-b0ae-304abbc2456f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:50.474025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f1437a-0587-4228-b0ae-304abbc2456f None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:50.475367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 212/426] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:50.476102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1437a-0587-4228-b0ae-304abbc2456f None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:50.547291 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf99d3-970f-437b-b021-0ec78e54f631 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:50.593842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe8c909-3bc8-464e-8744-9ccdacbfe527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:50.594832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8c909-3bc8-464e-8744-9ccdacbfe527 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:50.595118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8c909-3bc8-464e-8744-9ccdacbfe527 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:50.595398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe8c909-3bc8-464e-8744-9ccdacbfe527 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:50.596445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8c909-3bc8-464e-8744-9ccdacbfe527 None admin] RBAC: Adding query filter params (name=member) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:50.596758 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8c909-3bc8-464e-8744-9ccdacbfe527 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:50.599018 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe8c909-3bc8-464e-8744-9ccdacbfe527 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:50.611308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 215/427] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:37:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:50.612696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc046a-7f08-4df9-a555-9e00f082f565 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:50.613018 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc046a-7f08-4df9-a555-9e00f082f565 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:50.613306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc046a-7f08-4df9-a555-9e00f082f565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:50.635289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-c0cc046a-7f08-4df9-a555-9e00f082f565 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:50.636308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f1437a-0587-4228-b0ae-304abbc2456f None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:50.636589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c0cc046a-7f08-4df9-a555-9e00f082f565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:50.681333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:50.720459 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf96ff-b816-4799-9894-c12c61d9a16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:50.720459 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf96ff-b816-4799-9894-c12c61d9a16f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:50.720459 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf96ff-b816-4799-9894-c12c61d9a16f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:50.724038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf96ff-b816-4799-9894-c12c61d9a16f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:50.724038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf96ff-b816-4799-9894-c12c61d9a16f None admin] RBAC: Adding query filter params (group.id=50fa9d01d91a423bb20869ca7519fea0, role.id=fbd0f348273f4791b1813d1df24df10e, scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:50.724038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf96ff-b816-4799-9894-c12c61d9a16f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:50.728014 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf96ff-b816-4799-9894-c12c61d9a16f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:50.734231 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 216/428] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:50.735335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc046a-7f08-4df9-a555-9e00f082f565 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:50.750857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 213/429] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Jul 14 17:37:50 2022] GET /identity/v3/role_assignments?group.id=50fa9d01d91a423bb20869ca7519fea0&scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82&role.id=fbd0f348273f4791b1813d1df24df10e => generated 611 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:50.779963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf96ff-b816-4799-9894-c12c61d9a16f None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:50.812001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:50.969802 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf96ff-b816-4799-9894-c12c61d9a16f None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:51.032158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931d571-f455-4521-a99b-520d1d580ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:51.033581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931d571-f455-4521-a99b-520d1d580ce4 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:51.034170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931d571-f455-4521-a99b-520d1d580ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:51.034770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931d571-f455-4521-a99b-520d1d580ce4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:51.039234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc046a-7f08-4df9-a555-9e00f082f565 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:51.055604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e931d571-f455-4521-a99b-520d1d580ce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-e931d571-f455-4521-a99b-520d1d580ce4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:51.071942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:51.072963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:51.073785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:51.073785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:51.073785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:51.073785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:51.073785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:51.073785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:51.073785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:51.073785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:51.073785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:51.073785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:51.073785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:51.073785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:51.074997 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 214/430] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:37:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:51.084000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931d571-f455-4521-a99b-520d1d580ce4 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:51.104489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83580611-b148-4e79-b1d4-80ebae028e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:51.105290 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83580611-b148-4e79-b1d4-80ebae028e15 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:51.106016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83580611-b148-4e79-b1d4-80ebae028e15 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:51.106267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83580611-b148-4e79-b1d4-80ebae028e15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:51.156105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:51.183032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83580611-b148-4e79-b1d4-80ebae028e15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:51.185208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83580611-b148-4e79-b1d4-80ebae028e15 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:51.231864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:51.278574 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e931d571-f455-4521-a99b-520d1d580ce4 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:51.325700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 217/431] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:51.327716 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3da35b-e735-4f5c-bea4-d24ccf6d9003 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:51.328166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3da35b-e735-4f5c-bea4-d24ccf6d9003 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:51.328560 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3da35b-e735-4f5c-bea4-d24ccf6d9003 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:51.330434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a39244-4266-4c3e-8b11-753cccc9ad3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:51.331164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 218/432] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:51.331472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a39244-4266-4c3e-8b11-753cccc9ad3c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:51.331902 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a39244-4266-4c3e-8b11-753cccc9ad3c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:51.332358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a39244-4266-4c3e-8b11-753cccc9ad3c None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:51.332617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3da35b-e735-4f5c-bea4-d24ccf6d9003 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:51.335036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a39244-4266-4c3e-8b11-753cccc9ad3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:51.335362 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a39244-4266-4c3e-8b11-753cccc9ad3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:51.337156 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a39244-4266-4c3e-8b11-753cccc9ad3c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:51.348072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 215/433] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:37:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:51.349655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb6a89-b25e-48e7-995f-e1ba0bf5aa2b None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:51.349655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb6a89-b25e-48e7-995f-e1ba0bf5aa2b None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:51.349949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb6a89-b25e-48e7-995f-e1ba0bf5aa2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:51.368522 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-b7bb6a89-b25e-48e7-995f-e1ba0bf5aa2b None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:51.370616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b7bb6a89-b25e-48e7-995f-e1ba0bf5aa2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:51.427968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:51.452383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 216/434] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:51.453075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb6a89-b25e-48e7-995f-e1ba0bf5aa2b None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:51.481802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3da35b-e735-4f5c-bea4-d24ccf6d9003 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:51.524484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955a2dd-6e43-48c9-8989-56d39b06ac9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:51.525546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955a2dd-6e43-48c9-8989-56d39b06ac9d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:51.525910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955a2dd-6e43-48c9-8989-56d39b06ac9d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:51.526289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e955a2dd-6e43-48c9-8989-56d39b06ac9d None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:51.528282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955a2dd-6e43-48c9-8989-56d39b06ac9d None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:51.529856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e955a2dd-6e43-48c9-8989-56d39b06ac9d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:51.531894 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 219/435] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:51 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:51.536140 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955a2dd-6e43-48c9-8989-56d39b06ac9d None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:51.553784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:51.612952 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb6a89-b25e-48e7-995f-e1ba0bf5aa2b None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:51.664044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4034f-ae30-4b55-b6ef-fbd4a94bee5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:51.664704 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4034f-ae30-4b55-b6ef-fbd4a94bee5c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:51.664999 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4034f-ae30-4b55-b6ef-fbd4a94bee5c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:51.665267 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4034f-ae30-4b55-b6ef-fbd4a94bee5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:51.670029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e955a2dd-6e43-48c9-8989-56d39b06ac9d None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:51.724991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcb0cc-5f79-45b3-ae91-6459d8ce942f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:51.726233 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcb0cc-5f79-45b3-ae91-6459d8ce942f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:51.726667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcb0cc-5f79-45b3-ae91-6459d8ce942f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:51.727110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bcb0cc-5f79-45b3-ae91-6459d8ce942f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:51.732323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:51.747826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bcb0cc-5f79-45b3-ae91-6459d8ce942f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:51.751440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4034f-ae30-4b55-b6ef-fbd4a94bee5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:51.753133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4034f-ae30-4b55-b6ef-fbd4a94bee5c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-02bcb0cc-5f79-45b3-ae91-6459d8ce942f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:51.762926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:51.764333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:51.765418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:51.765418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:51.765418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:51.765418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:51.765418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:51.765418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:51.765418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:51.765418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:51.765418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:51.765418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:51.765418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:51.765418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:51.765418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 220/436] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:37:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:51.768110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcb0cc-5f79-45b3-ae91-6459d8ce942f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:51.857484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:51.903291 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 217/437] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:51.903291 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4034f-ae30-4b55-b6ef-fbd4a94bee5c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:51.926089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bcb0cc-5f79-45b3-ae91-6459d8ce942f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:51.984980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47c6db-dee8-498b-94d9-0880d0b6adfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:51.986383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47c6db-dee8-498b-94d9-0880d0b6adfb None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:51.986847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47c6db-dee8-498b-94d9-0880d0b6adfb None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:51.987306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad47c6db-dee8-498b-94d9-0880d0b6adfb None admin] PATH_INFO: `/v3/groups/50fa9d01d91a423bb20869ca7519fea0` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:51.989245 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47c6db-dee8-498b-94d9-0880d0b6adfb None admin] RBAC: Authorizing `identity:get_group(group_id=50fa9d01d91a423bb20869ca7519fea0)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:51.991743 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad47c6db-dee8-498b-94d9-0880d0b6adfb None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:51.994838 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 221/438] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:51 2022] GET /identity/v3/groups/50fa9d01d91a423bb20869ca7519fea0 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:51.996449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47c6db-dee8-498b-94d9-0880d0b6adfb None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:52.040197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4034f-ae30-4b55-b6ef-fbd4a94bee5c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:52.084151 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b6d15-711c-485c-a2ee-8de0378b4dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:52.084823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b6d15-711c-485c-a2ee-8de0378b4dcf None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:52.085245 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b6d15-711c-485c-a2ee-8de0378b4dcf None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:52.085520 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b6d15-711c-485c-a2ee-8de0378b4dcf None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:52.086402 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b6d15-711c-485c-a2ee-8de0378b4dcf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:52.086712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b6d15-711c-485c-a2ee-8de0378b4dcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:52.088129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0b6d15-711c-485c-a2ee-8de0378b4dcf None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:52.094551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:52.104113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 218/439] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:37:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:52.105134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b6d15-711c-485c-a2ee-8de0378b4dcf None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:52.147032 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:52.150418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad47c6db-dee8-498b-94d9-0880d0b6adfb None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:52.206174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a0be6-fdd8-4a1b-b4a1-33d12ec051d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:52.207686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a0be6-fdd8-4a1b-b4a1-33d12ec051d3 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:52.208604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a0be6-fdd8-4a1b-b4a1-33d12ec051d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:52.208604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a0be6-fdd8-4a1b-b4a1-33d12ec051d3 None admin] PATH_INFO: `/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:52.210089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a0be6-fdd8-4a1b-b4a1-33d12ec051d3 None admin] RBAC: Authorizing `identity:get_project(project_id=ba02b19bd80d44c4a9a9ae6517b11b82)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:52.212621 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a0be6-fdd8-4a1b-b4a1-33d12ec051d3 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:52.215614 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 222/440] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:51 2022] GET /identity/v3/projects/ba02b19bd80d44c4a9a9ae6517b11b82 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:52.221804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a0be6-fdd8-4a1b-b4a1-33d12ec051d3 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:52.254326 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0b6d15-711c-485c-a2ee-8de0378b4dcf None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:52.300236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1d637-7238-42ed-9869-679928555056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:52.301056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d637-7238-42ed-9869-679928555056 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:52.301540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d637-7238-42ed-9869-679928555056 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:52.301957 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a1d637-7238-42ed-9869-679928555056 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:52.303289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1d637-7238-42ed-9869-679928555056 None admin] RBAC: Adding query filter params (group.id=50fa9d01d91a423bb20869ca7519fea0, role.id=fbd0f348273f4791b1813d1df24df10e, scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:52.303724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1d637-7238-42ed-9869-679928555056 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:52.305525 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a1d637-7238-42ed-9869-679928555056 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:52.324002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 219/441] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Jul 14 17:37:52 2022] GET /identity/v3/role_assignments?group.id=50fa9d01d91a423bb20869ca7519fea0&scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2&role.id=fbd0f348273f4791b1813d1df24df10e => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:52.324002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1d637-7238-42ed-9869-679928555056 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:52.367726 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a0be6-fdd8-4a1b-b4a1-33d12ec051d3 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:52.421192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:52.429829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbfb4fd-0510-4678-8162-30551df06ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:52.430976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfb4fd-0510-4678-8162-30551df06ac1 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:52.431724 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfb4fd-0510-4678-8162-30551df06ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:52.432561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbfb4fd-0510-4678-8162-30551df06ac1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:52.498104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:52.508714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a1d637-7238-42ed-9869-679928555056 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:52.519101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbfb4fd-0510-4678-8162-30551df06ac1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:52.520735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbfb4fd-0510-4678-8162-30551df06ac1 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:52.577092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba874e3-b16b-4a69-94ed-0364a0849799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:52.580246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba874e3-b16b-4a69-94ed-0364a0849799 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:52.580246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba874e3-b16b-4a69-94ed-0364a0849799 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:52.580246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba874e3-b16b-4a69-94ed-0364a0849799 None admin] PATH_INFO: `/v3/groups/50fa9d01d91a423bb20869ca7519fea0` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:52.580673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba874e3-b16b-4a69-94ed-0364a0849799 None admin] RBAC: Authorizing `identity:get_group(group_id=50fa9d01d91a423bb20869ca7519fea0)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:52.583749 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba874e3-b16b-4a69-94ed-0364a0849799 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:52.585140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 220/442] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:52 2022] GET /identity/v3/groups/50fa9d01d91a423bb20869ca7519fea0 => generated 226 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:52.591604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba874e3-b16b-4a69-94ed-0364a0849799 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:52.653944 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 223/443] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:52.659730 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbfb4fd-0510-4678-8162-30551df06ac1 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:52.695335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:52.738521 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba874e3-b16b-4a69-94ed-0364a0849799 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:52.771107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbfb4fd-0510-4678-8162-30551df06ac1 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:52.772555 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9562a4-6da9-4b67-a7b3-a7590974cce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:52.773221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9562a4-6da9-4b67-a7b3-a7590974cce7 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:52.773541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9562a4-6da9-4b67-a7b3-a7590974cce7 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:52.773820 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9562a4-6da9-4b67-a7b3-a7590974cce7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:52.774679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9562a4-6da9-4b67-a7b3-a7590974cce7 None admin] RBAC: Adding query filter params (group.id=50fa9d01d91a423bb20869ca7519fea0, role.id=8427479f632a4da3949971f58b438c7f, scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:52.774993 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9562a4-6da9-4b67-a7b3-a7590974cce7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:52.776477 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9562a4-6da9-4b67-a7b3-a7590974cce7 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:52.783301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 221/444] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Jul 14 17:37:52 2022] GET /identity/v3/role_assignments?group.id=50fa9d01d91a423bb20869ca7519fea0&scope.project.id=ba02b19bd80d44c4a9a9ae6517b11b82&role.id=8427479f632a4da3949971f58b438c7f => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:52.784707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4def6-9644-4739-ab6b-da3c049a2e19 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:52.785000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4def6-9644-4739-ab6b-da3c049a2e19 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:52.785281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac4def6-9644-4739-ab6b-da3c049a2e19 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:52.786607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 222/445] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:52.797547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52afb57-74e9-4032-b360-762ee5f12573 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:52.798007 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52afb57-74e9-4032-b360-762ee5f12573 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:52.798463 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52afb57-74e9-4032-b360-762ee5f12573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:52.819749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a4beb-a2ab-47f4-af21-afb0bac1e43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:52.821080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a4beb-a2ab-47f4-af21-afb0bac1e43e None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:52.821486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a4beb-a2ab-47f4-af21-afb0bac1e43e None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:52.821874 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a4beb-a2ab-47f4-af21-afb0bac1e43e None admin] PATH_INFO: `/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:52.823784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a4beb-a2ab-47f4-af21-afb0bac1e43e None admin] RBAC: Authorizing `identity:get_project(project_id=1bebbd67fc624e7ea047e2d52573e5b2)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:52.825563 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-e52afb57-74e9-4032-b360-762ee5f12573 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:52.826368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a4beb-a2ab-47f4-af21-afb0bac1e43e None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:52.827456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e52afb57-74e9-4032-b360-762ee5f12573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:52.834400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 224/446] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:52 2022] GET /identity/v3/projects/1bebbd67fc624e7ea047e2d52573e5b2 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:52.836229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a4beb-a2ab-47f4-af21-afb0bac1e43e None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:52.862594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:52.947109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 223/447] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:52.973103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52afb57-74e9-4032-b360-762ee5f12573 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:53.003974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:53.008020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a4beb-a2ab-47f4-af21-afb0bac1e43e None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:53.063752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12b29a-0ced-4688-a1f0-825d362157e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:53.064824 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12b29a-0ced-4688-a1f0-825d362157e6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:53.065232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12b29a-0ced-4688-a1f0-825d362157e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:53.065616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12b29a-0ced-4688-a1f0-825d362157e6 None admin] PATH_INFO: `/v3/groups/50fa9d01d91a423bb20869ca7519fea0` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:53.067345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12b29a-0ced-4688-a1f0-825d362157e6 None admin] RBAC: Authorizing `identity:get_group(group_id=50fa9d01d91a423bb20869ca7519fea0)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:53.069982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12b29a-0ced-4688-a1f0-825d362157e6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:53.071736 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 225/448] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:52 2022] GET /identity/v3/groups/50fa9d01d91a423bb20869ca7519fea0 => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:53.078376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12b29a-0ced-4688-a1f0-825d362157e6 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:53.117940 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52afb57-74e9-4032-b360-762ee5f12573 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:53.167424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07748e-b358-4aff-8f52-f40bb84c2e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:53.169222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07748e-b358-4aff-8f52-f40bb84c2e60 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:53.169685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07748e-b358-4aff-8f52-f40bb84c2e60 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:53.170143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07748e-b358-4aff-8f52-f40bb84c2e60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:53.186504 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07748e-b358-4aff-8f52-f40bb84c2e60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-ac07748e-b358-4aff-8f52-f40bb84c2e60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:53.200077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:53.201079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:53.201865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:53.201865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:53.201865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:53.201865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:53.201865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:53.201865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:53.201865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:53.201865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:53.201865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:53.201865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:53.201865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:37:53.201865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:53.202464 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 224/449] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:37:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:37:53.209526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07748e-b358-4aff-8f52-f40bb84c2e60 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:53.223957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12b29a-0ced-4688-a1f0-825d362157e6 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:53.240008 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:53.276441 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19ec8f-7ead-4ab4-ac1d-63e838a6f851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:53.277131 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19ec8f-7ead-4ab4-ac1d-63e838a6f851 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:53.277409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19ec8f-7ead-4ab4-ac1d-63e838a6f851 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:53.277677 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19ec8f-7ead-4ab4-ac1d-63e838a6f851 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:53.278519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19ec8f-7ead-4ab4-ac1d-63e838a6f851 None admin] RBAC: Adding query filter params (group.id=50fa9d01d91a423bb20869ca7519fea0, role.id=8427479f632a4da3949971f58b438c7f, scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:53.278836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19ec8f-7ead-4ab4-ac1d-63e838a6f851 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:53.280330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19ec8f-7ead-4ab4-ac1d-63e838a6f851 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:53.288264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 226/450] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Jul 14 17:37:53 2022] GET /identity/v3/role_assignments?group.id=50fa9d01d91a423bb20869ca7519fea0&scope.project.id=1bebbd67fc624e7ea047e2d52573e5b2&role.id=8427479f632a4da3949971f58b438c7f => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:53.374355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07748e-b358-4aff-8f52-f40bb84c2e60 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:53.427817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c33edd-aeca-407b-8550-44e264c87196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:53.428917 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c33edd-aeca-407b-8550-44e264c87196 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:53.429400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c33edd-aeca-407b-8550-44e264c87196 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:53.429862 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c33edd-aeca-407b-8550-44e264c87196 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:53.431239 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c33edd-aeca-407b-8550-44e264c87196 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:53.431670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c33edd-aeca-407b-8550-44e264c87196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:53.434387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c33edd-aeca-407b-8550-44e264c87196 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:53.448184 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 225/451] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:37:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:37:53.456338 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19ec8f-7ead-4ab4-ac1d-63e838a6f851 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:53.493917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:53.578986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19ec8f-7ead-4ab4-ac1d-63e838a6f851 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:53.617050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63031629-1144-483f-85d5-79eaa3855562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:53.617725 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63031629-1144-483f-85d5-79eaa3855562 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:53.618007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63031629-1144-483f-85d5-79eaa3855562 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:53.618282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63031629-1144-483f-85d5-79eaa3855562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:53.694832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63031629-1144-483f-85d5-79eaa3855562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:53.696265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63031629-1144-483f-85d5-79eaa3855562 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:53.728169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:53.799660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 227/452] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:37:53.805406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c33edd-aeca-407b-8550-44e264c87196 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:53.882080 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:53.903427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c33edd-aeca-407b-8550-44e264c87196 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:53.936469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d042db-59b8-4467-887c-e2e75efbd406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:53.937326 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d042db-59b8-4467-887c-e2e75efbd406 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:53.937607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d042db-59b8-4467-887c-e2e75efbd406 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:53.937922 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d042db-59b8-4467-887c-e2e75efbd406 None admin] PATH_INFO: `/v3/projects/29b8bf26bc324367a4fe27e90fc2d6df` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:53.939094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d042db-59b8-4467-887c-e2e75efbd406 None admin] RBAC: Authorizing `identity:get_project(project_id=29b8bf26bc324367a4fe27e90fc2d6df)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:53.940578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d042db-59b8-4467-887c-e2e75efbd406 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:53.942273 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 226/453] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:37:53 2022] GET /identity/v3/projects/29b8bf26bc324367a4fe27e90fc2d6df => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:53.947884 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63031629-1144-483f-85d5-79eaa3855562 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:54.049158 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63031629-1144-483f-85d5-79eaa3855562 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:54.078239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:54.083546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f4b29e-a2e8-436b-92a2-bac2d5c4246e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:54.084433 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4b29e-a2e8-436b-92a2-bac2d5c4246e None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:54.084718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4b29e-a2e8-436b-92a2-bac2d5c4246e None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:54.084988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f4b29e-a2e8-436b-92a2-bac2d5c4246e None admin] PATH_INFO: `/v3/groups/a50abba7e46e484a8d50a3f54750b8f3` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:54.086136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4b29e-a2e8-436b-92a2-bac2d5c4246e None admin] RBAC: Authorizing `identity:get_group(group_id=a50abba7e46e484a8d50a3f54750b8f3)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:54.087567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f4b29e-a2e8-436b-92a2-bac2d5c4246e None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:54.089227 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 228/454] 10.222.0.64 () {56 vars in 1180 bytes} [Thu Jul 14 17:37:53 2022] GET /identity/v3/groups/a50abba7e46e484a8d50a3f54750b8f3 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:54.095902 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d042db-59b8-4467-887c-e2e75efbd406 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:54.198009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d042db-59b8-4467-887c-e2e75efbd406 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:54.231495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa7b932-7186-4b82-8eb2-52766e21f1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:54.232303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7b932-7186-4b82-8eb2-52766e21f1eb None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:54.232585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7b932-7186-4b82-8eb2-52766e21f1eb None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:54.232855 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa7b932-7186-4b82-8eb2-52766e21f1eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:54.233726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7b932-7186-4b82-8eb2-52766e21f1eb None admin] RBAC: Adding query filter params (group.id=a50abba7e46e484a8d50a3f54750b8f3, role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=29b8bf26bc324367a4fe27e90fc2d6df) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:54.234001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7b932-7186-4b82-8eb2-52766e21f1eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:54.235429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa7b932-7186-4b82-8eb2-52766e21f1eb None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:54.241320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 227/455] 10.222.0.64 () {56 vars in 1376 bytes} [Thu Jul 14 17:37:54 2022] GET /identity/v3/role_assignments?group.id=a50abba7e46e484a8d50a3f54750b8f3&scope.project.id=29b8bf26bc324367a4fe27e90fc2d6df&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:56.770432 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bbd741-7120-4f75-ae41-c2b36612662f None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:56.771455 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bbd741-7120-4f75-ae41-c2b36612662f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:56.771993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bbd741-7120-4f75-ae41-c2b36612662f None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:56.773844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 229/456] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:56.781560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9212896-94a4-4dd0-9d0f-7951e75cae09 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:56.782070 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9212896-94a4-4dd0-9d0f-7951e75cae09 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:56.782847 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9212896-94a4-4dd0-9d0f-7951e75cae09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:56.820963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-b9212896-94a4-4dd0-9d0f-7951e75cae09 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:56.822518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b9212896-94a4-4dd0-9d0f-7951e75cae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:56.826051 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:56.921905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 228/457] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:56.986311 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bbd741-7120-4f75-ae41-c2b36612662f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:57.262623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bbd741-7120-4f75-ae41-c2b36612662f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:57.326969 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d3ff6-f6f7-4ce2-a148-395f0ff9620a None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:57.327628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d3ff6-f6f7-4ce2-a148-395f0ff9620a None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:57.328166 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d3ff6-f6f7-4ce2-a148-395f0ff9620a None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:57.329988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 229/458] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:57.335278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bf922-7670-4ad1-85ee-247711222d6d None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:57.335823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bf922-7670-4ad1-85ee-247711222d6d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:57.336270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb92c5-8397-4c3c-b4b3-e60dbd8ec01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:57.336594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98bf922-7670-4ad1-85ee-247711222d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:57.337644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb92c5-8397-4c3c-b4b3-e60dbd8ec01b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:57.338162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb92c5-8397-4c3c-b4b3-e60dbd8ec01b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:57.339609 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb92c5-8397-4c3c-b4b3-e60dbd8ec01b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:57.351485 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb92c5-8397-4c3c-b4b3-e60dbd8ec01b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:57.352997 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb92c5-8397-4c3c-b4b3-e60dbd8ec01b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-e2eb92c5-8397-4c3c-b4b3-e60dbd8ec01b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:37:57.374569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:57.376288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:57.377983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:37:57.377983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:37:57.377983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:37:57.377983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:57.377983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 230/459] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:57.378538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-f98bf922-7670-4ad1-85ee-247711222d6d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:57.378538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f98bf922-7670-4ad1-85ee-247711222d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:57.388459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb92c5-8397-4c3c-b4b3-e60dbd8ec01b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:57.447895 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:57.468648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 230/460] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:57.470305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18fc76-a7dc-4dab-962b-a328d912e597 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:57.470767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18fc76-a7dc-4dab-962b-a328d912e597 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:57.471190 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18fc76-a7dc-4dab-962b-a328d912e597 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:57.472666 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 231/461] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:57.486485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a2ae3-c8e3-4792-8cec-02264c544c57 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:57.487004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a2ae3-c8e3-4792-8cec-02264c544c57 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:57.487457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2a2ae3-c8e3-4792-8cec-02264c544c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:57.513944 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-6c2a2ae3-c8e3-4792-8cec-02264c544c57 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:57.515629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6c2a2ae3-c8e3-4792-8cec-02264c544c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:57.522280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb92c5-8397-4c3c-b4b3-e60dbd8ec01b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:57.568160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b8188-93e0-4220-a251-08b17d04a101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:57.569487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b8188-93e0-4220-a251-08b17d04a101 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:57.570166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b8188-93e0-4220-a251-08b17d04a101 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:57.570594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10b8188-93e0-4220-a251-08b17d04a101 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:57.571647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b8188-93e0-4220-a251-08b17d04a101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:57.572133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b8188-93e0-4220-a251-08b17d04a101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:57.574900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10b8188-93e0-4220-a251-08b17d04a101 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:57.587655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:57.588080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 231/462] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:57.589496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b8188-93e0-4220-a251-08b17d04a101 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:57.603982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 232/463] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:37:57.603982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eeea0-93bb-4910-a8f5-feacf3ce9bb3 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:57.603982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eeea0-93bb-4910-a8f5-feacf3ce9bb3 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:57.603982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3eeea0-93bb-4910-a8f5-feacf3ce9bb3 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:57.603982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 233/464] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:57.603982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3eeea0-93bb-4910-a8f5-feacf3ce9bb3 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:57.709112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10b8188-93e0-4220-a251-08b17d04a101 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:57.710162 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3eeea0-93bb-4910-a8f5-feacf3ce9bb3 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:57.723619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:57.748205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f2651-50f1-4ab4-b73f-330a8ebeb714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:57.748894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f2651-50f1-4ab4-b73f-330a8ebeb714 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:57.749181 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f2651-50f1-4ab4-b73f-330a8ebeb714 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:57.749314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a24d2-da92-4329-adcf-ef4383f4fb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:57.749513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f2651-50f1-4ab4-b73f-330a8ebeb714 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:57.750040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a24d2-da92-4329-adcf-ef4383f4fb4c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:57.750172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f2651-50f1-4ab4-b73f-330a8ebeb714 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:57.750362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a24d2-da92-4329-adcf-ef4383f4fb4c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:57.750606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7a24d2-da92-4329-adcf-ef4383f4fb4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:57.751634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f2651-50f1-4ab4-b73f-330a8ebeb714 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:57.763526 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a24d2-da92-4329-adcf-ef4383f4fb4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:57.765182 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7a24d2-da92-4329-adcf-ef4383f4fb4c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:57.787964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 234/465] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:37:57 2022] POST /identity/v3/users => generated 246 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:37:57.787964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f04cd5-a19a-486c-90f7-06f3e4665928 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:57.787964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f04cd5-a19a-486c-90f7-06f3e4665928 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:57.787964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f04cd5-a19a-486c-90f7-06f3e4665928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-cb7a24d2-da92-4329-adcf-ef4383f4fb4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:37:57.788769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:57.790003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:57.791060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:37:57.791060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:37:57.791060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:37:57.791060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:57.793725 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 232/466] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:57.795054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a24d2-da92-4329-adcf-ef4383f4fb4c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:57.811886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-c9f04cd5-a19a-486c-90f7-06f3e4665928 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:57.813369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c9f04cd5-a19a-486c-90f7-06f3e4665928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:57.874523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:57.902868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 235/467] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:57.903893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f04cd5-a19a-486c-90f7-06f3e4665928 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:57.959321 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7a24d2-da92-4329-adcf-ef4383f4fb4c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:58.007152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7bd08-f6ec-4ab2-a1bd-8e19c1fd4a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:58.008023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7bd08-f6ec-4ab2-a1bd-8e19c1fd4a47 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:58.008290 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7bd08-f6ec-4ab2-a1bd-8e19c1fd4a47 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:58.008576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7bd08-f6ec-4ab2-a1bd-8e19c1fd4a47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:58.021249 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7bd08-f6ec-4ab2-a1bd-8e19c1fd4a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:58.022727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7bd08-f6ec-4ab2-a1bd-8e19c1fd4a47 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:58.039846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f04cd5-a19a-486c-90f7-06f3e4665928 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-5ea7bd08-f6ec-4ab2-a1bd-8e19c1fd4a47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:37:58.044793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:58.045686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:58.046544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:37:58.046544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:37:58.046544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:37:58.046544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:58.050995 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 233/468] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:58.051984 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7bd08-f6ec-4ab2-a1bd-8e19c1fd4a47 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:58.076275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:58.097483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f1c78-8661-4ed6-8907-42ea197eab19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:58.098808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f1c78-8661-4ed6-8907-42ea197eab19 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:58.098808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f1c78-8661-4ed6-8907-42ea197eab19 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:58.098808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f1c78-8661-4ed6-8907-42ea197eab19 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:58.099654 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f1c78-8661-4ed6-8907-42ea197eab19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:58.100057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f1c78-8661-4ed6-8907-42ea197eab19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:58.101604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6f1c78-8661-4ed6-8907-42ea197eab19 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:58.119948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 236/469] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:58.121829 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f1c78-8661-4ed6-8907-42ea197eab19 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:58.174171 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:58.193691 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7bd08-f6ec-4ab2-a1bd-8e19c1fd4a47 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:58.235691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6f1c78-8661-4ed6-8907-42ea197eab19 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:58.244377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938c58c-5c99-49c1-9392-48e50a2d81f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:58.245180 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938c58c-5c99-49c1-9392-48e50a2d81f7 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:58.245458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938c58c-5c99-49c1-9392-48e50a2d81f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:58.245988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938c58c-5c99-49c1-9392-48e50a2d81f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:58.257298 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938c58c-5c99-49c1-9392-48e50a2d81f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:58.258749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938c58c-5c99-49c1-9392-48e50a2d81f7 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-9938c58c-5c99-49c1-9392-48e50a2d81f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:37:58.278670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:58.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:58.280487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:37:58.280487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:37:58.280487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:37:58.280487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:58.284781 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 234/470] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:58.285782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938c58c-5c99-49c1-9392-48e50a2d81f7 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:58.286927 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4842d79-1195-4bc8-81ca-e8c4c4c60533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:58.287858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4842d79-1195-4bc8-81ca-e8c4c4c60533 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:58.288387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4842d79-1195-4bc8-81ca-e8c4c4c60533 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:58.289223 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4842d79-1195-4bc8-81ca-e8c4c4c60533 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:58.291302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4842d79-1195-4bc8-81ca-e8c4c4c60533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:58.291910 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4842d79-1195-4bc8-81ca-e8c4c4c60533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:58.293645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4842d79-1195-4bc8-81ca-e8c4c4c60533 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:58.308894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 237/471] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:58.310127 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4842d79-1195-4bc8-81ca-e8c4c4c60533 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:58.395562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:58.430348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4842d79-1195-4bc8-81ca-e8c4c4c60533 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:58.440582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:58.474397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650969fc-95ee-4b1b-8f45-7b7dfc66b6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:58.475502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650969fc-95ee-4b1b-8f45-7b7dfc66b6dd None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:58.476058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650969fc-95ee-4b1b-8f45-7b7dfc66b6dd None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:58.476350 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938c58c-5c99-49c1-9392-48e50a2d81f7 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:58.476764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650969fc-95ee-4b1b-8f45-7b7dfc66b6dd None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:58.477701 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650969fc-95ee-4b1b-8f45-7b7dfc66b6dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:58.478207 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650969fc-95ee-4b1b-8f45-7b7dfc66b6dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:58.479989 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650969fc-95ee-4b1b-8f45-7b7dfc66b6dd None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:58.493971 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 238/472] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:37:58.495223 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650969fc-95ee-4b1b-8f45-7b7dfc66b6dd None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:58.521537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ce032-0bb4-4447-b98d-a6ca045abe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:58.522225 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ce032-0bb4-4447-b98d-a6ca045abe27 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:58.522513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ce032-0bb4-4447-b98d-a6ca045abe27 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:58.522800 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ce032-0bb4-4447-b98d-a6ca045abe27 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:58.523612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ce032-0bb4-4447-b98d-a6ca045abe27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:58.525125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ce032-0bb4-4447-b98d-a6ca045abe27 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:58.550962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 235/473] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:37:58 2022] POST /identity/v3/users => generated 248 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:37:58.551880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ce032-0bb4-4447-b98d-a6ca045abe27 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:58.683557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650969fc-95ee-4b1b-8f45-7b7dfc66b6dd None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:58.708291 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ce032-0bb4-4447-b98d-a6ca045abe27 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:58.765272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:58.770036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445e1ae-88a0-411e-a49c-d53559a2ef16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:58.770896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445e1ae-88a0-411e-a49c-d53559a2ef16 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:58.771194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445e1ae-88a0-411e-a49c-d53559a2ef16 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:58.771479 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445e1ae-88a0-411e-a49c-d53559a2ef16 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:58.774435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445e1ae-88a0-411e-a49c-d53559a2ef16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:58.774435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445e1ae-88a0-411e-a49c-d53559a2ef16 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:58.782707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3b83d-e4c1-41dc-adb5-ddafa0bb493c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:58.783684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3b83d-e4c1-41dc-adb5-ddafa0bb493c None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:58.784317 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3b83d-e4c1-41dc-adb5-ddafa0bb493c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:58.784872 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3b83d-e4c1-41dc-adb5-ddafa0bb493c None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:58.786000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3b83d-e4c1-41dc-adb5-ddafa0bb493c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:58.788010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3b83d-e4c1-41dc-adb5-ddafa0bb493c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:58.804460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 239/474] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:37:58 2022] POST /identity/v3/users => generated 248 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:37:58.832432 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 236/475] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:37:58 2022] POST /identity/v3/users => generated 249 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:37:59.059681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67838763-5192-4092-bd5c-07b09a85e975 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:59.060218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67838763-5192-4092-bd5c-07b09a85e975 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:59.060727 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67838763-5192-4092-bd5c-07b09a85e975 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:59.062697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 240/476] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:37:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:37:59.068780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ec346-a81a-4234-86c4-17546afcc52c None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:59.069072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ec346-a81a-4234-86c4-17546afcc52c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:59.069351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ec346-a81a-4234-86c4-17546afcc52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:59.088389 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-652ec346-a81a-4234-86c4-17546afcc52c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:37:59.089659 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-652ec346-a81a-4234-86c4-17546afcc52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:37:59.174417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 237/477] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:37:59.197959 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67838763-5192-4092-bd5c-07b09a85e975 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:59.226073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:59.315231 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67838763-5192-4092-bd5c-07b09a85e975 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:59.351552 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6de156-80f8-4d38-9bd1-dc1558f0bedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:59.352253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6de156-80f8-4d38-9bd1-dc1558f0bedd None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:59.352541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6de156-80f8-4d38-9bd1-dc1558f0bedd None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:59.352820 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6de156-80f8-4d38-9bd1-dc1558f0bedd None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:59.353607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6de156-80f8-4d38-9bd1-dc1558f0bedd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:59.355007 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6de156-80f8-4d38-9bd1-dc1558f0bedd None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:59.362243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.sql.core [None req-6e6de156-80f8-4d38-9bd1-dc1558f0bedd None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 17:37:59.362243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 17:37:59.362243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [parameters: {'id': '43ca3433538546fda7aaf9a4ca19c4a5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 17:37:59.362243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91925) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-6e6de156-80f8-4d38-9bd1-dc1558f0bedd None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application conn.query(q) Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application result.read() Jul 14 17:37:59.380982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return ref.to_dict() Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 17:37:59.381925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application next(self.gen) Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application yield resource Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application next(self.gen) Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise self.value Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application session.commit() Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 17:37:59.382911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self.session.flush() Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application compat.raise_( Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 17:37:59.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 17:37:59.386411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application conn.query(q) Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application result.read() Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 17:37:59.391530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [parameters: {'id': '43ca3433538546fda7aaf9a4ca19c4a5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:59.392591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 17:37:59.393582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:37:59.394595 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 241/478] 10.222.0.64 () {60 vars in 1131 bytes} [Thu Jul 14 17:37:59 2022] POST /identity/v3/roles => generated 144 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:59.394672 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ec346-a81a-4234-86c4-17546afcc52c None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:59.526146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ec346-a81a-4234-86c4-17546afcc52c None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:59.541395 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:59.567813 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87493312-2ef4-4329-825d-a17c39f87cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:59.568852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87493312-2ef4-4329-825d-a17c39f87cea None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:59.569136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87493312-2ef4-4329-825d-a17c39f87cea None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:59.569440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87493312-2ef4-4329-825d-a17c39f87cea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:59.581266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87493312-2ef4-4329-825d-a17c39f87cea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-87493312-2ef4-4329-825d-a17c39f87cea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:37:59.589843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:37:59.591257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:37:59.592121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:37:59.592121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:37:59.592121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:37:59.592121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:37:59.592121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:37:59.592121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:37:59.592121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:37:59.592121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:37:59.592121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:37:59.592121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:37:59.592121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 17:37:59.592121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:37:59.592121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 238/479] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:37:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:37:59.597710 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6de156-80f8-4d38-9bd1-dc1558f0bedd None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:37:59.693830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:37:59.710374 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6de156-80f8-4d38-9bd1-dc1558f0bedd None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:37:59.749028 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6940a456-3933-4c44-9b85-5c534cf49376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:37:59.749706 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940a456-3933-4c44-9b85-5c534cf49376 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:37:59.749989 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940a456-3933-4c44-9b85-5c534cf49376 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:37:59.750266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940a456-3933-4c44-9b85-5c534cf49376 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:37:59.751026 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940a456-3933-4c44-9b85-5c534cf49376 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:37:59.751334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940a456-3933-4c44-9b85-5c534cf49376 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:37:59.752741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6940a456-3933-4c44-9b85-5c534cf49376 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:37:59.761965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 242/480] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:37:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:00.277758 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580ea51-ce30-4ebc-ae5b-51252db70c5b None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:00.278437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580ea51-ce30-4ebc-ae5b-51252db70c5b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:00.278437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580ea51-ce30-4ebc-ae5b-51252db70c5b None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:00.279968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 239/481] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:00.285609 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309b7d8-7318-4621-92c2-aed737e2c8e1 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:00.286170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309b7d8-7318-4621-92c2-aed737e2c8e1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:00.286170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309b7d8-7318-4621-92c2-aed737e2c8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:00.304949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4309b7d8-7318-4621-92c2-aed737e2c8e1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:00.306151 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4309b7d8-7318-4621-92c2-aed737e2c8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:00.400106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 243/482] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:00.440708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580ea51-ce30-4ebc-ae5b-51252db70c5b None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:00.560579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580ea51-ce30-4ebc-ae5b-51252db70c5b None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:00.612771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b340c1-19d6-4664-bcea-bf228283627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:00.613631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b340c1-19d6-4664-bcea-bf228283627f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:00.613916 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b340c1-19d6-4664-bcea-bf228283627f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:00.614192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b340c1-19d6-4664-bcea-bf228283627f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:00.631519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b340c1-19d6-4664-bcea-bf228283627f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-55b340c1-19d6-4664-bcea-bf228283627f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:00.639596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:00.640519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:00.641323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:00.641323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:00.641323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:00.641323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:00.641323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:00.641323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:00.641323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:00.641323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:00.641323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:00.641323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:00.641323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:00.641323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:00.641323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 240/483] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:00.647002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309b7d8-7318-4621-92c2-aed737e2c8e1 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:00.756298 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:00.781593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309b7d8-7318-4621-92c2-aed737e2c8e1 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:00.821753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcda5f-803e-4f9f-bcb8-c5added0b343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:00.822434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcda5f-803e-4f9f-bcb8-c5added0b343 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:00.822720 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcda5f-803e-4f9f-bcb8-c5added0b343 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:00.822998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcda5f-803e-4f9f-bcb8-c5added0b343 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:00.823764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcda5f-803e-4f9f-bcb8-c5added0b343 None admin] RBAC: Adding query filter params (name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:00.824103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcda5f-803e-4f9f-bcb8-c5added0b343 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:00.826177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcda5f-803e-4f9f-bcb8-c5added0b343 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:00.835690 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 244/484] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:00.841226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b340c1-19d6-4664-bcea-bf228283627f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:00.878789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:00.971946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b340c1-19d6-4664-bcea-bf228283627f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:01.026689 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354dc2a-fcbd-42ad-99b0-0b0ea8de4e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:01.028052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354dc2a-fcbd-42ad-99b0-0b0ea8de4e67 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:01.028365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354dc2a-fcbd-42ad-99b0-0b0ea8de4e67 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:01.028649 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9354dc2a-fcbd-42ad-99b0-0b0ea8de4e67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:01.049296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354dc2a-fcbd-42ad-99b0-0b0ea8de4e67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:01.050790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9354dc2a-fcbd-42ad-99b0-0b0ea8de4e67 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-9354dc2a-fcbd-42ad-99b0-0b0ea8de4e67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:01.075776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:01.076733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:01.077570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:01.077570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:01.077570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:01.077570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:01.077570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 241/485] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:01.089099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcda5f-803e-4f9f-bcb8-c5added0b343 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:01.232602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcda5f-803e-4f9f-bcb8-c5added0b343 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:01.268182 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471e89e8-40a8-4e27-8194-7bc7fb6e90a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:01.268848 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e89e8-40a8-4e27-8194-7bc7fb6e90a1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:01.269133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e89e8-40a8-4e27-8194-7bc7fb6e90a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:01.269410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471e89e8-40a8-4e27-8194-7bc7fb6e90a1 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:01.270143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e89e8-40a8-4e27-8194-7bc7fb6e90a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:01.270452 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e89e8-40a8-4e27-8194-7bc7fb6e90a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:01.271837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471e89e8-40a8-4e27-8194-7bc7fb6e90a1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:01.285353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 245/486] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:01.285620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:01.290761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354dc2a-fcbd-42ad-99b0-0b0ea8de4e67 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:01.322403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2a638-9f9d-41c1-9d16-2d766f21b881 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:01.323375 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2a638-9f9d-41c1-9d16-2d766f21b881 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:01.323865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2a638-9f9d-41c1-9d16-2d766f21b881 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:01.326524 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 246/487] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:38:01.332403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db278852-1494-403d-af8b-52b20059c1a2 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:01.333236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db278852-1494-403d-af8b-52b20059c1a2 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:01.333723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db278852-1494-403d-af8b-52b20059c1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:01.353850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-db278852-1494-403d-af8b-52b20059c1a2 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:01.355402 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-db278852-1494-403d-af8b-52b20059c1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:01.444298 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:01.475898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9354dc2a-fcbd-42ad-99b0-0b0ea8de4e67 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:01.483964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 247/488] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:01.483964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f1d7d-fd58-43f4-a53d-dce26a4802c7 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:01.483964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f1d7d-fd58-43f4-a53d-dce26a4802c7 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:01.483964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f1d7d-fd58-43f4-a53d-dce26a4802c7 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:01.483964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 248/489] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:01.493515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5e725-efb5-47a9-b088-0066b9e58d70 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:01.493803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5e725-efb5-47a9-b088-0066b9e58d70 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:01.494083 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5e725-efb5-47a9-b088-0066b9e58d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:01.519667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-50c5e725-efb5-47a9-b088-0066b9e58d70 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:01.521202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-50c5e725-efb5-47a9-b088-0066b9e58d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:01.536862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938919bb-7bd8-4565-8b3c-75639e407a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:01.538319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938919bb-7bd8-4565-8b3c-75639e407a1a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:01.538822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938919bb-7bd8-4565-8b3c-75639e407a1a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:01.539579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938919bb-7bd8-4565-8b3c-75639e407a1a None admin] PATH_INFO: `/v3/users/nova` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:01.550953 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938919bb-7bd8-4565-8b3c-75639e407a1a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:01.553667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938919bb-7bd8-4565-8b3c-75639e407a1a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-938919bb-7bd8-4565-8b3c-75639e407a1a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:01.574778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:01.575957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:01.577121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:01.577121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:01.577121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:01.577121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:01.577121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:01.577121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:01.577121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:01.577121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:01.577121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:38:01.577121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:01.579861 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 242/490] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Jul 14 17:38:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:01.581339 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938919bb-7bd8-4565-8b3c-75639e407a1a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:01.612022 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 249/491] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:38:01.613147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5e725-efb5-47a9-b088-0066b9e58d70 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:01.687048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:01.708921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938919bb-7bd8-4565-8b3c-75639e407a1a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:01.732331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5e725-efb5-47a9-b088-0066b9e58d70 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:01.748487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4d9bf-54e9-4fe9-a75d-1b8d2b0d0a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:01.749317 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4d9bf-54e9-4fe9-a75d-1b8d2b0d0a55 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:01.749722 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4d9bf-54e9-4fe9-a75d-1b8d2b0d0a55 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:01.750101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f4d9bf-54e9-4fe9-a75d-1b8d2b0d0a55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:01.762404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f4d9bf-54e9-4fe9-a75d-1b8d2b0d0a55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-86f4d9bf-54e9-4fe9-a75d-1b8d2b0d0a55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:01.770665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:01.771626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:01.772466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:01.772466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:01.772466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:01.772466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:01.772466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:01.772466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:01.772466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:01.772466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:01.772466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:01.772466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:01.772466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:01.772466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:01.773851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9417ea-9c77-495f-ad2d-aa7f447a4ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:01.774781 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9417ea-9c77-495f-ad2d-aa7f447a4ebe None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:01.775262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9417ea-9c77-495f-ad2d-aa7f447a4ebe None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:01.775743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9417ea-9c77-495f-ad2d-aa7f447a4ebe None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:01.776823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9417ea-9c77-495f-ad2d-aa7f447a4ebe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:01.777201 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 243/492] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:01.778126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9417ea-9c77-495f-ad2d-aa7f447a4ebe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:01.778657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4d9bf-54e9-4fe9-a75d-1b8d2b0d0a55 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:01.780054 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9417ea-9c77-495f-ad2d-aa7f447a4ebe None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:01.782386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:01.820855 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 250/493] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Jul 14 17:38:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:01.821377 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9417ea-9c77-495f-ad2d-aa7f447a4ebe None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:01.893161 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f4d9bf-54e9-4fe9-a75d-1b8d2b0d0a55 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:01.925759 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9417ea-9c77-495f-ad2d-aa7f447a4ebe None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:01.930264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d83c1-2f93-41cd-a887-99bb114bf5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:01.931247 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d83c1-2f93-41cd-a887-99bb114bf5f6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:01.931675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d83c1-2f93-41cd-a887-99bb114bf5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:01.932143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622d83c1-2f93-41cd-a887-99bb114bf5f6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:01.944373 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622d83c1-2f93-41cd-a887-99bb114bf5f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:01.944826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-622d83c1-2f93-41cd-a887-99bb114bf5f6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:01.952652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:01.953674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:01.954479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:01.954479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:01.954479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:01.954479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:01.954479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:01.954479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:01.954479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:01.954479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:01.954479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:01.954479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:01.954479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:01.954479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:01.954479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 244/494] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:01.955229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d83c1-2f93-41cd-a887-99bb114bf5f6 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:01.971122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e78e0d4-1e8d-4ee1-98f2-dd04d5cbbaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:01.971812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78e0d4-1e8d-4ee1-98f2-dd04d5cbbaa3 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:01.972128 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78e0d4-1e8d-4ee1-98f2-dd04d5cbbaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:01.972416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e78e0d4-1e8d-4ee1-98f2-dd04d5cbbaa3 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:01.973362 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78e0d4-1e8d-4ee1-98f2-dd04d5cbbaa3 None admin] RBAC: Adding query filter params (name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:01.973681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78e0d4-1e8d-4ee1-98f2-dd04d5cbbaa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:01.975091 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e78e0d4-1e8d-4ee1-98f2-dd04d5cbbaa3 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:01.984308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 251/495] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:01.985711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2082-523e-4043-b800-f6bc5f4b82a7 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:01.986009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2082-523e-4043-b800-f6bc5f4b82a7 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:01.986291 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375a2082-523e-4043-b800-f6bc5f4b82a7 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:01.991618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 252/496] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:01.991618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a2082-523e-4043-b800-f6bc5f4b82a7 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:02.034035 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:02.079751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622d83c1-2f93-41cd-a887-99bb114bf5f6 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:02.109095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375a2082-523e-4043-b800-f6bc5f4b82a7 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:02.124212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f03fa99-f952-41c6-8322-f5f3e9300f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:02.125144 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03fa99-f952-41c6-8322-f5f3e9300f57 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:02.125430 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03fa99-f952-41c6-8322-f5f3e9300f57 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:02.125714 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03fa99-f952-41c6-8322-f5f3e9300f57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:02.149381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526dab2-3437-4581-ad68-54bd45dce288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:02.150056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526dab2-3437-4581-ad68-54bd45dce288 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:02.150340 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526dab2-3437-4581-ad68-54bd45dce288 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:02.150612 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526dab2-3437-4581-ad68-54bd45dce288 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:02.151368 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526dab2-3437-4581-ad68-54bd45dce288 None admin] RBAC: Adding query filter params (name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:02.151673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526dab2-3437-4581-ad68-54bd45dce288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:02.153192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526dab2-3437-4581-ad68-54bd45dce288 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:02.162517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 253/497] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:02.165851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526dab2-3437-4581-ad68-54bd45dce288 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:02.207020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03fa99-f952-41c6-8322-f5f3e9300f57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:02.212717 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f03fa99-f952-41c6-8322-f5f3e9300f57 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:02.217098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:02.283078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:02.291618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526dab2-3437-4581-ad68-54bd45dce288 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:02.332468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18640ab-95d3-4a9e-9c8f-764baeaeced6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:02.333198 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18640ab-95d3-4a9e-9c8f-764baeaeced6 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:02.333549 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18640ab-95d3-4a9e-9c8f-764baeaeced6 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:02.333834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18640ab-95d3-4a9e-9c8f-764baeaeced6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:02.336238 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 245/498] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:38:02.338508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b858-88fc-4480-9885-bad00531b93c None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:02.339014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b858-88fc-4480-9885-bad00531b93c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:02.339474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b858-88fc-4480-9885-bad00531b93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:02.348214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18640ab-95d3-4a9e-9c8f-764baeaeced6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:02.349242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18640ab-95d3-4a9e-9c8f-764baeaeced6 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:02.359308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-9999b858-88fc-4480-9885-bad00531b93c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:02.360531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9999b858-88fc-4480-9885-bad00531b93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-f18640ab-95d3-4a9e-9c8f-764baeaeced6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:02.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:02.368665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:02.369560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:02.369560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:02.369560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:02.369560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:02.369560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 254/499] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:02.369818 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18640ab-95d3-4a9e-9c8f-764baeaeced6 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:02.433415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 246/500] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:02.434842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999b858-88fc-4480-9885-bad00531b93c None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:02.476416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:02.484007 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18640ab-95d3-4a9e-9c8f-764baeaeced6 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:02.516863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a5a4d-1ce6-4254-abca-c3c36942c6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:02.517651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a5a4d-1ce6-4254-abca-c3c36942c6d2 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:02.517930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a5a4d-1ce6-4254-abca-c3c36942c6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:02.518206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54a5a4d-1ce6-4254-abca-c3c36942c6d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:02.528608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a5a4d-1ce6-4254-abca-c3c36942c6d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:02.530047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54a5a4d-1ce6-4254-abca-c3c36942c6d2 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:02.538517 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-a54a5a4d-1ce6-4254-abca-c3c36942c6d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:02.547870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:02.548746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:02.549591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:02.549591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:02.549591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:02.549591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:02.549799 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999b858-88fc-4480-9885-bad00531b93c None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:02.549866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 255/501] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:02.549866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a5a4d-1ce6-4254-abca-c3c36942c6d2 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:02.582969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f07261-24eb-4c8a-a704-ff25a25c2f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:02.583686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f07261-24eb-4c8a-a704-ff25a25c2f03 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:02.584007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f07261-24eb-4c8a-a704-ff25a25c2f03 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:02.584284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f07261-24eb-4c8a-a704-ff25a25c2f03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:02.594545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f07261-24eb-4c8a-a704-ff25a25c2f03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:02.596134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f07261-24eb-4c8a-a704-ff25a25c2f03 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-d3f07261-24eb-4c8a-a704-ff25a25c2f03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:02.614285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:02.615341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:02.622028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:02.622028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:02.622028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:02.622028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:02.622028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 247/502] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:02.622028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad2c28-a843-4b73-af32-ad26a4d2f1ac None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:02.622028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad2c28-a843-4b73-af32-ad26a4d2f1ac None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:02.622028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ad2c28-a843-4b73-af32-ad26a4d2f1ac None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:02.622028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 248/503] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:02.622618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad2c28-a843-4b73-af32-ad26a4d2f1ac None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:02.652933 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54a5a4d-1ce6-4254-abca-c3c36942c6d2 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:02.687101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2548b1-c9fa-4b97-b0c3-1f4fed9ab785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:02.687770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2548b1-c9fa-4b97-b0c3-1f4fed9ab785 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:02.688100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2548b1-c9fa-4b97-b0c3-1f4fed9ab785 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:02.688383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2548b1-c9fa-4b97-b0c3-1f4fed9ab785 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:02.689119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2548b1-c9fa-4b97-b0c3-1f4fed9ab785 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:02.689447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2548b1-c9fa-4b97-b0c3-1f4fed9ab785 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:02.691111 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2548b1-c9fa-4b97-b0c3-1f4fed9ab785 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:02.703851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 256/504] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:02.704724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2548b1-c9fa-4b97-b0c3-1f4fed9ab785 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:02.714243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:02.723135 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ad2c28-a843-4b73-af32-ad26a4d2f1ac None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:02.757989 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8872ee-b0d5-4d86-b48a-4d1da7243e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:02.758787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8872ee-b0d5-4d86-b48a-4d1da7243e94 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:02.759079 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8872ee-b0d5-4d86-b48a-4d1da7243e94 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:02.759367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8872ee-b0d5-4d86-b48a-4d1da7243e94 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:02.769620 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8872ee-b0d5-4d86-b48a-4d1da7243e94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-6f8872ee-b0d5-4d86-b48a-4d1da7243e94 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:02.777534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:02.778553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:02.779328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:02.779328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:02.779328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:02.779328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:02.779328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:02.779328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:02.779328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:02.779328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:02.779328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:02.779328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:02.779328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:02.779328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:02.779328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 249/505] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:02.780008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8872ee-b0d5-4d86-b48a-4d1da7243e94 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:02.789401 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:02.807012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2548b1-c9fa-4b97-b0c3-1f4fed9ab785 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:02.841344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81041c18-f857-419e-9293-073cedb2fb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:02.841995 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81041c18-f857-419e-9293-073cedb2fb21 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:02.842297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81041c18-f857-419e-9293-073cedb2fb21 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:02.842569 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81041c18-f857-419e-9293-073cedb2fb21 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:02.843301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81041c18-f857-419e-9293-073cedb2fb21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:02.843613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81041c18-f857-419e-9293-073cedb2fb21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:02.845044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81041c18-f857-419e-9293-073cedb2fb21 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:02.856889 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 257/506] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:02.858149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc925a-e230-4b79-ac00-58d5aa3fc4bd None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:02.858440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc925a-e230-4b79-ac00-58d5aa3fc4bd None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:02.858724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc925a-e230-4b79-ac00-58d5aa3fc4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:02.875969 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-d1dc925a-e230-4b79-ac00-58d5aa3fc4bd None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:02.877187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d1dc925a-e230-4b79-ac00-58d5aa3fc4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:02.881903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8872ee-b0d5-4d86-b48a-4d1da7243e94 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:02.915214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6aa0f-e48f-4b20-a529-02f770694495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:02.915887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6aa0f-e48f-4b20-a529-02f770694495 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:02.916202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6aa0f-e48f-4b20-a529-02f770694495 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:02.916479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6aa0f-e48f-4b20-a529-02f770694495 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:02.917204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6aa0f-e48f-4b20-a529-02f770694495 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:02.917515 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6aa0f-e48f-4b20-a529-02f770694495 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:02.919425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c6aa0f-e48f-4b20-a529-02f770694495 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:02.931273 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 250/507] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:02.932112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6aa0f-e48f-4b20-a529-02f770694495 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:02.936866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:02.944879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 258/508] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:02.945715 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc925a-e230-4b79-ac00-58d5aa3fc4bd None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:02.996856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:03.034690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6aa0f-e48f-4b20-a529-02f770694495 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:03.057584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc925a-e230-4b79-ac00-58d5aa3fc4bd None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:03.070910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a356f29-e376-4413-993b-d707e9eab90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:03.071713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a356f29-e376-4413-993b-d707e9eab90d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:03.072034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a356f29-e376-4413-993b-d707e9eab90d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:03.072313 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a356f29-e376-4413-993b-d707e9eab90d None admin] PATH_INFO: `/v3/users/glance` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:03.079665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a356f29-e376-4413-993b-d707e9eab90d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:03.081151 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a356f29-e376-4413-993b-d707e9eab90d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:03.095194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623a30f-ddf7-4640-984a-1296b0fd7d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:03.095194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623a30f-ddf7-4640-984a-1296b0fd7d15 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:03.095194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623a30f-ddf7-4640-984a-1296b0fd7d15 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:03.095194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0623a30f-ddf7-4640-984a-1296b0fd7d15 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:03.095523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623a30f-ddf7-4640-984a-1296b0fd7d15 None admin] RBAC: Adding query filter params (name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:03.095523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623a30f-ddf7-4640-984a-1296b0fd7d15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-7a356f29-e376-4413-993b-d707e9eab90d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:03.096164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:03.096983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:03.097933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:03.097933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:03.097933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:03.097933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:03.097933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:03.097933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:03.097933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:03.097933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:03.097933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 17:38:03.097933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:03.098619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0623a30f-ddf7-4640-984a-1296b0fd7d15 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:03.098699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 251/509] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:38:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:03.099424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a356f29-e376-4413-993b-d707e9eab90d None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:03.107324 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 259/510] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:03.108504 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623a30f-ddf7-4640-984a-1296b0fd7d15 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:03.167551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:03.200575 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a356f29-e376-4413-993b-d707e9eab90d None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:03.210809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0623a30f-ddf7-4640-984a-1296b0fd7d15 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:03.223355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:03.235461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15326c-5aaf-45e1-87a8-6c58a277d7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:03.236516 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15326c-5aaf-45e1-87a8-6c58a277d7cd None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:03.236806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15326c-5aaf-45e1-87a8-6c58a277d7cd None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:03.237087 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15326c-5aaf-45e1-87a8-6c58a277d7cd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:03.244728 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15326c-5aaf-45e1-87a8-6c58a277d7cd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:03.247276 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f15326c-5aaf-45e1-87a8-6c58a277d7cd None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:03.247730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4db060-5517-42c5-8741-8c3663f903ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:03.248559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4db060-5517-42c5-8741-8c3663f903ec None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:03.248834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4db060-5517-42c5-8741-8c3663f903ec None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:03.249114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4db060-5517-42c5-8741-8c3663f903ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:03.256050 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4db060-5517-42c5-8741-8c3663f903ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:03.257473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4db060-5517-42c5-8741-8c3663f903ec None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-3f15326c-5aaf-45e1-87a8-6c58a277d7cd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:03.262794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:03.263757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:03.264686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:03.264686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:03.264686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:03.264686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:03.264686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:03.264686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:03.264686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:03.264686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:03.264686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 17:38:03.264686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:03.264686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 252/511] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:03.265241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15326c-5aaf-45e1-87a8-6c58a277d7cd None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-6e4db060-5517-42c5-8741-8c3663f903ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:03.277107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:03.278608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:03.279441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:03.279441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:03.279441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:03.279441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:03.279441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:03.279441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:03.279441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:03.279441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:03.279441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 260/512] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:03.280364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4db060-5517-42c5-8741-8c3663f903ec None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:03.377349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f15326c-5aaf-45e1-87a8-6c58a277d7cd None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:03.381757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4db060-5517-42c5-8741-8c3663f903ec None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:03.399506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:03.411192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b5bd7-a0ee-49ac-b6f0-607e8255ecb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:03.412025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b5bd7-a0ee-49ac-b6f0-607e8255ecb3 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:03.412313 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b5bd7-a0ee-49ac-b6f0-607e8255ecb3 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:03.412591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b5bd7-a0ee-49ac-b6f0-607e8255ecb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:03.416112 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea62e4-1d3f-4085-8fec-dd29fd676346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:03.416778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea62e4-1d3f-4085-8fec-dd29fd676346 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:03.417059 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea62e4-1d3f-4085-8fec-dd29fd676346 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:03.417333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea62e4-1d3f-4085-8fec-dd29fd676346 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:03.418869 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea62e4-1d3f-4085-8fec-dd29fd676346 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:03.419219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea62e4-1d3f-4085-8fec-dd29fd676346 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:03.420691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea62e4-1d3f-4085-8fec-dd29fd676346 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:03.423350 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b5bd7-a0ee-49ac-b6f0-607e8255ecb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:03.424812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b5bd7-a0ee-49ac-b6f0-607e8255ecb3 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:03.438633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 261/513] 10.222.0.64 () {56 vars in 1137 bytes} [Thu Jul 14 17:38:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:03.439471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea62e4-1d3f-4085-8fec-dd29fd676346 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-291b5bd7-a0ee-49ac-b6f0-607e8255ecb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:03.443335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:03.444232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:03.445083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:03.445083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:03.445083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:03.445083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:03.445083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 253/514] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:03.445342 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b5bd7-a0ee-49ac-b6f0-607e8255ecb3 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:03.478653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:03.544497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea62e4-1d3f-4085-8fec-dd29fd676346 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:03.544752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b5bd7-a0ee-49ac-b6f0-607e8255ecb3 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:03.578593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90730bac-0dc2-4038-a9ff-55a7c26738b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:03.579111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576bb3f4-9865-43c1-9072-bc976ae40b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:03.579383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90730bac-0dc2-4038-a9ff-55a7c26738b2 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:03.579663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90730bac-0dc2-4038-a9ff-55a7c26738b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:03.579788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bb3f4-9865-43c1-9072-bc976ae40b93 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:03.579954 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90730bac-0dc2-4038-a9ff-55a7c26738b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:03.580105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bb3f4-9865-43c1-9072-bc976ae40b93 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:03.580385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576bb3f4-9865-43c1-9072-bc976ae40b93 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:03.581229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576bb3f4-9865-43c1-9072-bc976ae40b93 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:03.581544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576bb3f4-9865-43c1-9072-bc976ae40b93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:03.582981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576bb3f4-9865-43c1-9072-bc976ae40b93 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:03.590158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90730bac-0dc2-4038-a9ff-55a7c26738b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:03.591623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90730bac-0dc2-4038-a9ff-55a7c26738b2 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-90730bac-0dc2-4038-a9ff-55a7c26738b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:03.609652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:03.610732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:03.611745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:03.611745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:03.611745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:03.611745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:03.611745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 262/515] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:03.612106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90730bac-0dc2-4038-a9ff-55a7c26738b2 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:03.620435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 254/516] 10.222.0.64 () {56 vars in 1139 bytes} [Thu Jul 14 17:38:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:03.622097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576bb3f4-9865-43c1-9072-bc976ae40b93 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:03.640527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:03.714388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90730bac-0dc2-4038-a9ff-55a7c26738b2 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:03.718409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:03.725708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576bb3f4-9865-43c1-9072-bc976ae40b93 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:03.749475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f35fe-62f1-43d6-a2e7-64e255d30546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:03.750131 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f35fe-62f1-43d6-a2e7-64e255d30546 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:03.750414 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f35fe-62f1-43d6-a2e7-64e255d30546 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:03.750684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f35fe-62f1-43d6-a2e7-64e255d30546 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:03.751470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f35fe-62f1-43d6-a2e7-64e255d30546 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:03.751777 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f35fe-62f1-43d6-a2e7-64e255d30546 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:03.753250 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f35fe-62f1-43d6-a2e7-64e255d30546 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:03.759957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865628cb-1bd7-4ddc-acf8-e6ea94feee98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:03.760615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865628cb-1bd7-4ddc-acf8-e6ea94feee98 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:03.760901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865628cb-1bd7-4ddc-acf8-e6ea94feee98 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:03.761178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865628cb-1bd7-4ddc-acf8-e6ea94feee98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:03.771451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 263/517] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:03.772306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f35fe-62f1-43d6-a2e7-64e255d30546 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:03.829559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865628cb-1bd7-4ddc-acf8-e6ea94feee98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:03.831051 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865628cb-1bd7-4ddc-acf8-e6ea94feee98 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:03.860431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:03.873721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f35fe-62f1-43d6-a2e7-64e255d30546 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:03.907594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d1aba-3448-422e-a646-1b0e0d9f5e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:03.908212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d1aba-3448-422e-a646-1b0e0d9f5e55 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:03.908767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d1aba-3448-422e-a646-1b0e0d9f5e55 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:03.908767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260d1aba-3448-422e-a646-1b0e0d9f5e55 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:03.909494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d1aba-3448-422e-a646-1b0e0d9f5e55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:03.909808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d1aba-3448-422e-a646-1b0e0d9f5e55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:03.911183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260d1aba-3448-422e-a646-1b0e0d9f5e55 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:03.922450 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 264/518] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:03.923266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d1aba-3448-422e-a646-1b0e0d9f5e55 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:03.925557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:03.934692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 255/519] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:38:03.935524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865628cb-1bd7-4ddc-acf8-e6ea94feee98 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:04.028834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260d1aba-3448-422e-a646-1b0e0d9f5e55 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:04.035127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865628cb-1bd7-4ddc-acf8-e6ea94feee98 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:04.062569 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410d94c-b705-42c5-9d36-5a928541bd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:04.063234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410d94c-b705-42c5-9d36-5a928541bd27 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:04.063522 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410d94c-b705-42c5-9d36-5a928541bd27 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:04.063799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410d94c-b705-42c5-9d36-5a928541bd27 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:04.064562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410d94c-b705-42c5-9d36-5a928541bd27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:04.064874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410d94c-b705-42c5-9d36-5a928541bd27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:04.066239 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7410d94c-b705-42c5-9d36-5a928541bd27 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:04.068492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8cb62-c620-4fea-8a1a-1ecefc61e63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:04.069147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8cb62-c620-4fea-8a1a-1ecefc61e63a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:04.069439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8cb62-c620-4fea-8a1a-1ecefc61e63a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:04.069720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8cb62-c620-4fea-8a1a-1ecefc61e63a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:04.078565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 265/520] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:04.079381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410d94c-b705-42c5-9d36-5a928541bd27 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:04.089802 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:04.131809 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:04.140412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8cb62-c620-4fea-8a1a-1ecefc61e63a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:04.141892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8cb62-c620-4fea-8a1a-1ecefc61e63a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:04.183723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410d94c-b705-42c5-9d36-5a928541bd27 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:04.219457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fea95b-b962-4d42-a33c-3532d62f015c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:04.220143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fea95b-b962-4d42-a33c-3532d62f015c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:04.220432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fea95b-b962-4d42-a33c-3532d62f015c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:04.220710 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fea95b-b962-4d42-a33c-3532d62f015c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:04.221557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fea95b-b962-4d42-a33c-3532d62f015c None admin] RBAC: Adding query filter params (role.id=2db2bc363aa74930932c823dc291d7f3, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=6423f54152064b909d03c0908d636993) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:04.221876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fea95b-b962-4d42-a33c-3532d62f015c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:04.223384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fea95b-b962-4d42-a33c-3532d62f015c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:04.229212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 266/521] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:04 2022] GET /identity/v3/role_assignments?user.id=6423f54152064b909d03c0908d636993&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=2db2bc363aa74930932c823dc291d7f3 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:04.229978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fea95b-b962-4d42-a33c-3532d62f015c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:04.245445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 256/522] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:38:04.247558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8cb62-c620-4fea-8a1a-1ecefc61e63a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:04.327722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:04.363356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:04.367347 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fea95b-b962-4d42-a33c-3532d62f015c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:04.373797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8cb62-c620-4fea-8a1a-1ecefc61e63a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:04.406936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b33c0-18e9-4de2-9fd8-a098595bbe36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:04.407724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b33c0-18e9-4de2-9fd8-a098595bbe36 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:04.408099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b33c0-18e9-4de2-9fd8-a098595bbe36 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:04.408552 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b33c0-18e9-4de2-9fd8-a098595bbe36 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:04.409234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b33c0-18e9-4de2-9fd8-a098595bbe36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:04.410771 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b33c0-18e9-4de2-9fd8-a098595bbe36 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:04.420022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813884ef-905d-4712-8a36-5fa931793257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:04.420943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813884ef-905d-4712-8a36-5fa931793257 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:04.421226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813884ef-905d-4712-8a36-5fa931793257 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:04.421502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813884ef-905d-4712-8a36-5fa931793257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:04.432885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 267/523] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:38:04 2022] POST /identity/v3/users => generated 247 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:38:04.433114 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813884ef-905d-4712-8a36-5fa931793257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:04.434790 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b33c0-18e9-4de2-9fd8-a098595bbe36 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:04.435325 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813884ef-905d-4712-8a36-5fa931793257 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-813884ef-905d-4712-8a36-5fa931793257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:04.459737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:04.460798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:04.461858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:04.461858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:04.461858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:04.461858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:04.466438 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 257/524] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:04.467593 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813884ef-905d-4712-8a36-5fa931793257 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:04.589118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813884ef-905d-4712-8a36-5fa931793257 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:04.618914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b33c0-18e9-4de2-9fd8-a098595bbe36 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:04.645061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d9499-71fe-4796-b37a-f6e3d2e01658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:04.645896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d9499-71fe-4796-b37a-f6e3d2e01658 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:04.646180 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d9499-71fe-4796-b37a-f6e3d2e01658 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:04.646459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d9499-71fe-4796-b37a-f6e3d2e01658 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:04.648083 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:04.663072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d9499-71fe-4796-b37a-f6e3d2e01658 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:04.666743 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814d9499-71fe-4796-b37a-f6e3d2e01658 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:04.666951 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0431e-c72b-4e11-80b2-d409b9f2a965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:04.669066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0431e-c72b-4e11-80b2-d409b9f2a965 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:04.669066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0431e-c72b-4e11-80b2-d409b9f2a965 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:04.669066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0431e-c72b-4e11-80b2-d409b9f2a965 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:04.673893 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:04.680876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0431e-c72b-4e11-80b2-d409b9f2a965 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:04.682533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0431e-c72b-4e11-80b2-d409b9f2a965 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-814d9499-71fe-4796-b37a-f6e3d2e01658 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:04.689289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:04.690423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:04.691548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:04.691548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:04.691548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:04.691548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:04.691548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 258/525] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:04.691896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d9499-71fe-4796-b37a-f6e3d2e01658 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-1ae0431e-c72b-4e11-80b2-d409b9f2a965 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:04.705753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:04.706916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:04.708711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:04.708711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:04.708711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:04.708711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:04.708711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:04.708711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:04.708711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:04.708711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:04.708711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 17:38:04.708711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:04.708711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 268/526] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:38:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:04.710218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0431e-c72b-4e11-80b2-d409b9f2a965 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:04.832639 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0431e-c72b-4e11-80b2-d409b9f2a965 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:04.840775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d9499-71fe-4796-b37a-f6e3d2e01658 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:04.879244 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe4b94-491f-48b6-894f-c3146a4f4023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:04.880733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe4b94-491f-48b6-894f-c3146a4f4023 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:04.881312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe4b94-491f-48b6-894f-c3146a4f4023 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:04.881839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafe4b94-491f-48b6-894f-c3146a4f4023 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:04.882732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe4b94-491f-48b6-894f-c3146a4f4023 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:04.883284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe4b94-491f-48b6-894f-c3146a4f4023 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:04.885072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafe4b94-491f-48b6-894f-c3146a4f4023 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:04.885381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682656b5-6c8c-4946-8851-c2035d400983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:04.886170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682656b5-6c8c-4946-8851-c2035d400983 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:04.886559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682656b5-6c8c-4946-8851-c2035d400983 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:04.887028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682656b5-6c8c-4946-8851-c2035d400983 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:04.887886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682656b5-6c8c-4946-8851-c2035d400983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:04.888391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682656b5-6c8c-4946-8851-c2035d400983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:04.889921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682656b5-6c8c-4946-8851-c2035d400983 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:04.902446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 269/527] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:04.903962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe4b94-491f-48b6-894f-c3146a4f4023 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:04.907806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 259/528] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:04.909268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682656b5-6c8c-4946-8851-c2035d400983 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:04.971597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:04.992062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:05.038457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafe4b94-491f-48b6-894f-c3146a4f4023 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:05.043650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682656b5-6c8c-4946-8851-c2035d400983 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:05.081195 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334c3d0-942e-45f4-85ce-081965f468fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:05.082020 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334c3d0-942e-45f4-85ce-081965f468fe None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:05.082429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334c3d0-942e-45f4-85ce-081965f468fe None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:05.082723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334c3d0-942e-45f4-85ce-081965f468fe None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:05.083673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334c3d0-942e-45f4-85ce-081965f468fe None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:05.084021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334c3d0-942e-45f4-85ce-081965f468fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:05.085461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334c3d0-942e-45f4-85ce-081965f468fe None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:05.087780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742d528-4a9b-4915-abd9-cf95ff745de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:05.088961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742d528-4a9b-4915-abd9-cf95ff745de0 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:05.089487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742d528-4a9b-4915-abd9-cf95ff745de0 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:05.090060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742d528-4a9b-4915-abd9-cf95ff745de0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:05.098108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742d528-4a9b-4915-abd9-cf95ff745de0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:05.099886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742d528-4a9b-4915-abd9-cf95ff745de0 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:05.108827 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 270/529] 10.222.0.64 () {56 vars in 1137 bytes} [Thu Jul 14 17:38:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:05.109753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334c3d0-942e-45f4-85ce-081965f468fe None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-5742d528-4a9b-4915-abd9-cf95ff745de0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:05.117056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:05.117976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:05.118821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:05.118821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:05.118821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:05.118821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:05.118821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:05.118821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:05.118821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:05.118821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:05.127793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 260/530] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:05.129045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742d528-4a9b-4915-abd9-cf95ff745de0 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:05.248046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334c3d0-942e-45f4-85ce-081965f468fe None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:05.250181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742d528-4a9b-4915-abd9-cf95ff745de0 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:05.256750 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:05.279267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:05.290589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1df834-3570-43ea-8a2b-8b799a7f3d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:05.291721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1df834-3570-43ea-8a2b-8b799a7f3d2d None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:05.292271 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1df834-3570-43ea-8a2b-8b799a7f3d2d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:05.292753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1df834-3570-43ea-8a2b-8b799a7f3d2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:05.300843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1df834-3570-43ea-8a2b-8b799a7f3d2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:05.302557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7cef0-cd7b-408d-8282-c94f18209e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:05.302982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1df834-3570-43ea-8a2b-8b799a7f3d2d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:05.303221 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7cef0-cd7b-408d-8282-c94f18209e1a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:05.303508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7cef0-cd7b-408d-8282-c94f18209e1a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:05.303791 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7cef0-cd7b-408d-8282-c94f18209e1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-da1df834-3570-43ea-8a2b-8b799a7f3d2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:05.319804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:05.320787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:05.321659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:05.321659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:05.321659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:05.321659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:05.321659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:05.321659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:05.321659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:05.321659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:05.329887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 271/531] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:05.329887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1df834-3570-43ea-8a2b-8b799a7f3d2d None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:05.386035 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7cef0-cd7b-408d-8282-c94f18209e1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:05.388150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7cef0-cd7b-408d-8282-c94f18209e1a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:05.462273 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1df834-3570-43ea-8a2b-8b799a7f3d2d None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:05.494500 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 261/532] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:38:05.495373 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7cef0-cd7b-408d-8282-c94f18209e1a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:05.506375 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4259c-4f4d-4db9-bdf7-3ccc1b424755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:05.506980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4259c-4f4d-4db9-bdf7-3ccc1b424755 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:05.507587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4259c-4f4d-4db9-bdf7-3ccc1b424755 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:05.507876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4259c-4f4d-4db9-bdf7-3ccc1b424755 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:05.509010 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4259c-4f4d-4db9-bdf7-3ccc1b424755 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:05.509333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4259c-4f4d-4db9-bdf7-3ccc1b424755 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:05.511334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4259c-4f4d-4db9-bdf7-3ccc1b424755 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:05.516926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:05.533074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 272/533] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:05.534918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4259c-4f4d-4db9-bdf7-3ccc1b424755 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:05.582802 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:05.617529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7cef0-cd7b-408d-8282-c94f18209e1a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:05.656544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf92222-6bca-44ea-9e8a-f1b60fe8d880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:05.657225 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf92222-6bca-44ea-9e8a-f1b60fe8d880 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:05.657524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf92222-6bca-44ea-9e8a-f1b60fe8d880 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:05.657867 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf92222-6bca-44ea-9e8a-f1b60fe8d880 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:05.658709 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf92222-6bca-44ea-9e8a-f1b60fe8d880 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:05.659031 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf92222-6bca-44ea-9e8a-f1b60fe8d880 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:05.660531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf92222-6bca-44ea-9e8a-f1b60fe8d880 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:05.684579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 262/534] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:05.685784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf92222-6bca-44ea-9e8a-f1b60fe8d880 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:05.693323 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4259c-4f4d-4db9-bdf7-3ccc1b424755 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:05.741721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0500bd-b1ba-433f-b299-d475fbbac6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:05.743180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0500bd-b1ba-433f-b299-d475fbbac6b9 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:05.743621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0500bd-b1ba-433f-b299-d475fbbac6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:05.744094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0500bd-b1ba-433f-b299-d475fbbac6b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:05.760524 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0500bd-b1ba-433f-b299-d475fbbac6b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:05.763102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0500bd-b1ba-433f-b299-d475fbbac6b9 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:05.774293 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-7b0500bd-b1ba-433f-b299-d475fbbac6b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:05.788420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:05.789503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:05.790554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:05.790554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:05.790554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:05.790554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:05.792901 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 273/535] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:05.794512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0500bd-b1ba-433f-b299-d475fbbac6b9 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:05.806681 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf92222-6bca-44ea-9e8a-f1b60fe8d880 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:05.855577 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2974a-70b4-4d02-8363-b072fb10d34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:05.857707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2974a-70b4-4d02-8363-b072fb10d34d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:05.858149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2974a-70b4-4d02-8363-b072fb10d34d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:05.858848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2974a-70b4-4d02-8363-b072fb10d34d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:05.860116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2974a-70b4-4d02-8363-b072fb10d34d None admin] RBAC: Adding query filter params (role.id=2db2bc363aa74930932c823dc291d7f3, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=a5adec9924444e29b8964f7969452592) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:05.860588 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2974a-70b4-4d02-8363-b072fb10d34d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:05.862887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2974a-70b4-4d02-8363-b072fb10d34d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:05.872216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 263/536] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:05 2022] GET /identity/v3/role_assignments?user.id=a5adec9924444e29b8964f7969452592&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=2db2bc363aa74930932c823dc291d7f3 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:05.873337 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2974a-70b4-4d02-8363-b072fb10d34d None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:05.898915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:05.956031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0500bd-b1ba-433f-b299-d475fbbac6b9 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:06.010309 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2974a-70b4-4d02-8363-b072fb10d34d None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:06.040870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d733962-89cd-4fe2-ba1c-730731f4e20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:06.041909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d733962-89cd-4fe2-ba1c-730731f4e20a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:06.041909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d733962-89cd-4fe2-ba1c-730731f4e20a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:06.042374 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d733962-89cd-4fe2-ba1c-730731f4e20a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:06.043513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d733962-89cd-4fe2-ba1c-730731f4e20a None admin] RBAC: Adding query filter params (role.id=2db2bc363aa74930932c823dc291d7f3, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=b2b4bf2aa1744a46b6d2a1b5a136a0e1) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:06.043859 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d733962-89cd-4fe2-ba1c-730731f4e20a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:06.045456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d733962-89cd-4fe2-ba1c-730731f4e20a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:06.059519 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 274/537] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:05 2022] GET /identity/v3/role_assignments?user.id=b2b4bf2aa1744a46b6d2a1b5a136a0e1&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=2db2bc363aa74930932c823dc291d7f3 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:06.060510 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afc32c-adde-415d-bbb6-d306be19fc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:06.061412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afc32c-adde-415d-bbb6-d306be19fc85 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:06.061878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afc32c-adde-415d-bbb6-d306be19fc85 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:06.062334 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afc32c-adde-415d-bbb6-d306be19fc85 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:06.063277 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afc32c-adde-415d-bbb6-d306be19fc85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:06.063801 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afc32c-adde-415d-bbb6-d306be19fc85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:06.065482 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afc32c-adde-415d-bbb6-d306be19fc85 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:06.081049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 264/538] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:06.089102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d733962-89cd-4fe2-ba1c-730731f4e20a None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:06.278703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d733962-89cd-4fe2-ba1c-730731f4e20a None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:06.316339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab1f18-97cb-4ad1-80bb-928cdce31113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:06.317090 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab1f18-97cb-4ad1-80bb-928cdce31113 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:06.317372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab1f18-97cb-4ad1-80bb-928cdce31113 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:06.317648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab1f18-97cb-4ad1-80bb-928cdce31113 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:06.328246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab1f18-97cb-4ad1-80bb-928cdce31113 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:06.329757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab1f18-97cb-4ad1-80bb-928cdce31113 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-39ab1f18-97cb-4ad1-80bb-928cdce31113 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:06.345534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:06.346716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:06.347562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:06.347562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:06.347562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:06.347562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:06.347562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:06.347562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:06.347562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:06.347562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:06.347562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 275/539] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:06.352936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afc32c-adde-415d-bbb6-d306be19fc85 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:06.356648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:06.492123 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afc32c-adde-415d-bbb6-d306be19fc85 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:06.531100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b7266-af4e-4b9d-8357-afcafa493d5d None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:06.531651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b7266-af4e-4b9d-8357-afcafa493d5d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:06.531651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5b7266-af4e-4b9d-8357-afcafa493d5d None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:06.536116 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 276/540] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:06.539322 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22b560-3e8e-4e89-8947-ce7fcf07fc2c None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:06.539619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22b560-3e8e-4e89-8947-ce7fcf07fc2c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:06.539905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b22b560-3e8e-4e89-8947-ce7fcf07fc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:06.551243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:06.558244 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657cf2c-d102-475a-bb7f-f995a771ee7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:06.559505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657cf2c-d102-475a-bb7f-f995a771ee7f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:06.560066 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657cf2c-d102-475a-bb7f-f995a771ee7f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:06.560360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6657cf2c-d102-475a-bb7f-f995a771ee7f None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:06.561217 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657cf2c-d102-475a-bb7f-f995a771ee7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:06.561548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657cf2c-d102-475a-bb7f-f995a771ee7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:06.563335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6657cf2c-d102-475a-bb7f-f995a771ee7f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:06.579654 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-6b22b560-3e8e-4e89-8947-ce7fcf07fc2c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:06.580839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6b22b560-3e8e-4e89-8947-ce7fcf07fc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:06.593706 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 265/541] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:06.600718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657cf2c-d102-475a-bb7f-f995a771ee7f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:06.709497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 277/542] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:38:06.718846 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:06.734572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22b560-3e8e-4e89-8947-ce7fcf07fc2c None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:06.783741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6657cf2c-d102-475a-bb7f-f995a771ee7f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:06.833381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c18a313-5d03-4aa5-88e9-dd9acd1d39da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:06.835022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18a313-5d03-4aa5-88e9-dd9acd1d39da None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:06.835428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18a313-5d03-4aa5-88e9-dd9acd1d39da None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:06.835813 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c18a313-5d03-4aa5-88e9-dd9acd1d39da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:06.836815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18a313-5d03-4aa5-88e9-dd9acd1d39da None admin] RBAC: Adding query filter params (role.id=2db2bc363aa74930932c823dc291d7f3, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=e49fad7db3c643b99e9071d9ca5f0196) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:06.837237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18a313-5d03-4aa5-88e9-dd9acd1d39da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:06.838939 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c18a313-5d03-4aa5-88e9-dd9acd1d39da None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:06.848783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 266/543] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:06 2022] GET /identity/v3/role_assignments?user.id=e49fad7db3c643b99e9071d9ca5f0196&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=2db2bc363aa74930932c823dc291d7f3 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:06.850579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8945dc6-b640-425c-afe6-b83cd86aedc6 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:06.850997 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8945dc6-b640-425c-afe6-b83cd86aedc6 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:06.851383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8945dc6-b640-425c-afe6-b83cd86aedc6 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:06.856415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 267/544] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:06.860507 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49204cb6-da50-4f1e-be2d-ffce778403e8 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:06.861104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49204cb6-da50-4f1e-be2d-ffce778403e8 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:06.861443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49204cb6-da50-4f1e-be2d-ffce778403e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:06.879205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b22b560-3e8e-4e89-8947-ce7fcf07fc2c None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:06.905088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-49204cb6-da50-4f1e-be2d-ffce778403e8 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:06.906446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-49204cb6-da50-4f1e-be2d-ffce778403e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:06.914761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:06.922476 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd868d-cb2b-4936-9c58-05e2a455ab2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:06.923698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd868d-cb2b-4936-9c58-05e2a455ab2c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:06.924216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd868d-cb2b-4936-9c58-05e2a455ab2c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:06.924666 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd868d-cb2b-4936-9c58-05e2a455ab2c None admin] PATH_INFO: `/v3/roles/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:06.937864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd868d-cb2b-4936-9c58-05e2a455ab2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-3ccd868d-cb2b-4936-9c58-05e2a455ab2c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:06.946516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:06.947478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:06.948318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:06.948318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:06.948318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:06.948318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:06.948318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:06.948318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:06.948318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:06.948318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:06.948318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:06.948318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:06.948318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:06.948318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:06.951305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 278/545] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:06.959245 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd868d-cb2b-4936-9c58-05e2a455ab2c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:07.030071 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 268/546] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:07.056855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49204cb6-da50-4f1e-be2d-ffce778403e8 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:07.132305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:07.181555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49204cb6-da50-4f1e-be2d-ffce778403e8 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:07.211686 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd868d-cb2b-4936-9c58-05e2a455ab2c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:07.221125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6633c5-3bf8-4eee-aba7-5354dcb24356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:07.222072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6633c5-3bf8-4eee-aba7-5354dcb24356 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:07.222362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6633c5-3bf8-4eee-aba7-5354dcb24356 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:07.222741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6633c5-3bf8-4eee-aba7-5354dcb24356 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:07.236172 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6633c5-3bf8-4eee-aba7-5354dcb24356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-bc6633c5-3bf8-4eee-aba7-5354dcb24356 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:07.244669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:07.245586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:07.246680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:07.246680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:07.246680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:07.246680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:07.246680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:07.246680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:07.246680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:07.246680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:07.246680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:07.246680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:07.246680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:07.246680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:07.246680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 269/547] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:07.255988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6633c5-3bf8-4eee-aba7-5354dcb24356 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:07.261341 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825a1cb-dc71-4dc1-bbb3-4416302ce165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:07.262553 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825a1cb-dc71-4dc1-bbb3-4416302ce165 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:07.262836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825a1cb-dc71-4dc1-bbb3-4416302ce165 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:07.263107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825a1cb-dc71-4dc1-bbb3-4416302ce165 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:07.263865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825a1cb-dc71-4dc1-bbb3-4416302ce165 None admin] RBAC: Adding query filter params (name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:07.264221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825a1cb-dc71-4dc1-bbb3-4416302ce165 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:07.265620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825a1cb-dc71-4dc1-bbb3-4416302ce165 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:07.279593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 279/548] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:07.280881 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825a1cb-dc71-4dc1-bbb3-4416302ce165 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:07.396302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:07.401113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6633c5-3bf8-4eee-aba7-5354dcb24356 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:07.419275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:07.443431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4ab07-fe37-4ac5-9fec-6c850471d993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:07.447004 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4ab07-fe37-4ac5-9fec-6c850471d993 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:07.447543 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4ab07-fe37-4ac5-9fec-6c850471d993 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:07.448054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4ab07-fe37-4ac5-9fec-6c850471d993 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:07.451105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4ab07-fe37-4ac5-9fec-6c850471d993 None admin] RBAC: Adding query filter params (name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:07.451666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4ab07-fe37-4ac5-9fec-6c850471d993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:07.453366 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4ab07-fe37-4ac5-9fec-6c850471d993 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:07.477684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 270/549] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:07.485406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4ab07-fe37-4ac5-9fec-6c850471d993 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:07.506258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825a1cb-dc71-4dc1-bbb3-4416302ce165 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:07.577722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90921479-9e5b-476e-8c4f-785d2a8c99de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:07.578515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90921479-9e5b-476e-8c4f-785d2a8c99de None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:07.578710 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90921479-9e5b-476e-8c4f-785d2a8c99de None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:07.578981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90921479-9e5b-476e-8c4f-785d2a8c99de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:07.607514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4ab07-fe37-4ac5-9fec-6c850471d993 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:07.609073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90921479-9e5b-476e-8c4f-785d2a8c99de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:07.610577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90921479-9e5b-476e-8c4f-785d2a8c99de None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:07.649585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9f3a1-e585-4976-9905-79e78aab22cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:07.650308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9f3a1-e585-4976-9905-79e78aab22cd None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:07.650595 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9f3a1-e585-4976-9905-79e78aab22cd None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:07.650877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9f3a1-e585-4976-9905-79e78aab22cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-90921479-9e5b-476e-8c4f-785d2a8c99de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:07.656642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:07.657651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:07.658499 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:07.658499 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:07.658499 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:07.658499 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:07.663348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9f3a1-e585-4976-9905-79e78aab22cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:07.664379 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 280/550] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:07.664842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9f3a1-e585-4976-9905-79e78aab22cd None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:07.670634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90921479-9e5b-476e-8c4f-785d2a8c99de None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-e3c9f3a1-e585-4976-9905-79e78aab22cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:07.684488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:07.685392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:07.687104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:07.687104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:07.687104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:07.687104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:07.687104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 271/551] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:07.692624 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9f3a1-e585-4976-9905-79e78aab22cd None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:07.750474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:07.828296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9f3a1-e585-4976-9905-79e78aab22cd None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:07.850278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90921479-9e5b-476e-8c4f-785d2a8c99de None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:07.865177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5f221-77d9-4bba-b44c-40a50d7499fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:07.865860 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5f221-77d9-4bba-b44c-40a50d7499fe None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:07.866151 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5f221-77d9-4bba-b44c-40a50d7499fe None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:07.866431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5f221-77d9-4bba-b44c-40a50d7499fe None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:07.867165 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5f221-77d9-4bba-b44c-40a50d7499fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:07.867475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5f221-77d9-4bba-b44c-40a50d7499fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:07.868892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5f221-77d9-4bba-b44c-40a50d7499fe None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:07.869433 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:07.883904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 272/552] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:07.892975 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5f221-77d9-4bba-b44c-40a50d7499fe None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:07.912695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49b1ef-97c3-4bd3-9e6d-59d00f91897b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:07.915702 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49b1ef-97c3-4bd3-9e6d-59d00f91897b None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:07.916366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49b1ef-97c3-4bd3-9e6d-59d00f91897b None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:07.917925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49b1ef-97c3-4bd3-9e6d-59d00f91897b None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:07.919847 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49b1ef-97c3-4bd3-9e6d-59d00f91897b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:07.920578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49b1ef-97c3-4bd3-9e6d-59d00f91897b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:07.922508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49b1ef-97c3-4bd3-9e6d-59d00f91897b None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:07.947382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 281/553] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:07.953599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49b1ef-97c3-4bd3-9e6d-59d00f91897b None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:08.061996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49b1ef-97c3-4bd3-9e6d-59d00f91897b None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:08.100555 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b71df1-e81c-45ef-a8dd-cfe42b01d155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:08.101695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b71df1-e81c-45ef-a8dd-cfe42b01d155 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:08.102207 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b71df1-e81c-45ef-a8dd-cfe42b01d155 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:08.102688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b71df1-e81c-45ef-a8dd-cfe42b01d155 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:08.110919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b71df1-e81c-45ef-a8dd-cfe42b01d155 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:08.112726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b71df1-e81c-45ef-a8dd-cfe42b01d155 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:08.114779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-14b71df1-e81c-45ef-a8dd-cfe42b01d155 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:08.130661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:08.131668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:08.132726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:08.132726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:08.132726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:08.132726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:08.132726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:08.132726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:08.132726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:08.132726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:08.132726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:38:08.132726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:08.135013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 282/554] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Jul 14 17:38:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:08.140987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b71df1-e81c-45ef-a8dd-cfe42b01d155 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:08.154715 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5f221-77d9-4bba-b44c-40a50d7499fe None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:08.176965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:08.222253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc5dab5-d0ea-4ef9-8c6e-fd90f8919558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:08.224322 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5dab5-d0ea-4ef9-8c6e-fd90f8919558 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:08.227868 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5dab5-d0ea-4ef9-8c6e-fd90f8919558 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:08.228346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc5dab5-d0ea-4ef9-8c6e-fd90f8919558 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:08.243216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5dab5-d0ea-4ef9-8c6e-fd90f8919558 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:08.247023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc5dab5-d0ea-4ef9-8c6e-fd90f8919558 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-8cc5dab5-d0ea-4ef9-8c6e-fd90f8919558 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:08.269571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:08.270756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:08.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:08.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:08.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:08.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:08.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:08.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:08.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:08.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:08.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 17:38:08.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:08.275474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 273/555] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:38:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:08.283313 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25cfd9-3268-49d7-ad30-52eaafbb549c None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:08.284281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25cfd9-3268-49d7-ad30-52eaafbb549c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:08.285022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25cfd9-3268-49d7-ad30-52eaafbb549c None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:08.287042 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 274/556] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:08 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:08.287987 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25cfd9-3268-49d7-ad30-52eaafbb549c None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:08.315963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b71df1-e81c-45ef-a8dd-cfe42b01d155 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:08.360109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a0d7c8-152d-4109-95c8-b62c4deb6ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:08.361104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0d7c8-152d-4109-95c8-b62c4deb6ad7 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:08.361446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0d7c8-152d-4109-95c8-b62c4deb6ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:08.361747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a0d7c8-152d-4109-95c8-b62c4deb6ad7 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:08.362939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a0d7c8-152d-4109-95c8-b62c4deb6ad7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:08.363726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a0d7c8-152d-4109-95c8-b62c4deb6ad7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:08.365799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a0d7c8-152d-4109-95c8-b62c4deb6ad7 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:08.404032 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 283/557] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Jul 14 17:38:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:08.405542 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3badb-955d-4e95-8783-1ae81d6b8871 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:08.405862 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3badb-955d-4e95-8783-1ae81d6b8871 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:08.406138 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3badb-955d-4e95-8783-1ae81d6b8871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:08.422331 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25cfd9-3268-49d7-ad30-52eaafbb549c None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:08.423912 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-31a3badb-955d-4e95-8783-1ae81d6b8871 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:08.425371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-31a3badb-955d-4e95-8783-1ae81d6b8871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:08.467486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdd168-d006-44fe-be2f-f19d732a339c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:08.468231 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdd168-d006-44fe-be2f-f19d732a339c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:08.468518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdd168-d006-44fe-be2f-f19d732a339c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:08.468795 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fdd168-d006-44fe-be2f-f19d732a339c None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:08.469658 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fdd168-d006-44fe-be2f-f19d732a339c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:08.470135 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fdd168-d006-44fe-be2f-f19d732a339c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:08.471838 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fdd168-d006-44fe-be2f-f19d732a339c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:08.494280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 275/558] 10.222.0.64 () {56 vars in 1135 bytes} [Thu Jul 14 17:38:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:08.495189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdd168-d006-44fe-be2f-f19d732a339c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:08.506887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:08.524405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 284/559] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:08.524978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3badb-955d-4e95-8783-1ae81d6b8871 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:08.552569 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:08.620306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fdd168-d006-44fe-be2f-f19d732a339c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:08.642768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3badb-955d-4e95-8783-1ae81d6b8871 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:08.659680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d049c0-6587-4983-b5e4-d6aee0d26a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:08.660628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d049c0-6587-4983-b5e4-d6aee0d26a58 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:08.660911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d049c0-6587-4983-b5e4-d6aee0d26a58 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:08.661185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d049c0-6587-4983-b5e4-d6aee0d26a58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:08.674443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d049c0-6587-4983-b5e4-d6aee0d26a58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:08.675888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d049c0-6587-4983-b5e4-d6aee0d26a58 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:08.677625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728645f3-d6c0-479d-ad27-9628e8a8b503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:08.678281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728645f3-d6c0-479d-ad27-9628e8a8b503 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:08.678574 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728645f3-d6c0-479d-ad27-9628e8a8b503 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:08.678858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728645f3-d6c0-479d-ad27-9628e8a8b503 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-c7d049c0-6587-4983-b5e4-d6aee0d26a58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:08.694579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:08.695690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:08.696771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:08.696771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:08.696771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:08.696771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:08.698299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 276/560] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:08.704241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d049c0-6587-4983-b5e4-d6aee0d26a58 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:08.750738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728645f3-d6c0-479d-ad27-9628e8a8b503 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:08.752246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728645f3-d6c0-479d-ad27-9628e8a8b503 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:08.767890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:08.770064 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:08.817877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d049c0-6587-4983-b5e4-d6aee0d26a58 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:08.857623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a11e6-5082-46b5-8a5b-e88e959d624f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:08.858433 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a11e6-5082-46b5-8a5b-e88e959d624f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:08.858732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a11e6-5082-46b5-8a5b-e88e959d624f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:08.858988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a11e6-5082-46b5-8a5b-e88e959d624f None admin] PATH_INFO: `/v3/roles/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:08.867861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 285/561] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:38:08.867861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada597e4-fa0f-4132-8354-79bec596d0a7 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:08.867861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada597e4-fa0f-4132-8354-79bec596d0a7 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:08.867861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada597e4-fa0f-4132-8354-79bec596d0a7 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:08.867861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 286/562] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:08.868445 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada597e4-fa0f-4132-8354-79bec596d0a7 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:08.877405 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a11e6-5082-46b5-8a5b-e88e959d624f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-bd2a11e6-5082-46b5-8a5b-e88e959d624f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:08.885615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:08.891705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:08.892749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:08.892749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:08.892749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:08.892749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:08.892749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:08.892749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:08.892749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:08.892749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:08.892749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:08.892749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:08.892749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:08.892749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:08.892749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 277/563] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:08.892749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a11e6-5082-46b5-8a5b-e88e959d624f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:08.989997 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada597e4-fa0f-4132-8354-79bec596d0a7 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:09.005809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:09.007409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a11e6-5082-46b5-8a5b-e88e959d624f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:09.026599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ebad5-6744-486d-aade-f9768038c205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:09.027270 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ebad5-6744-486d-aade-f9768038c205 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:09.027680 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ebad5-6744-486d-aade-f9768038c205 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:09.028006 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ebad5-6744-486d-aade-f9768038c205 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:09.028745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ebad5-6744-486d-aade-f9768038c205 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:09.029056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ebad5-6744-486d-aade-f9768038c205 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:09.030505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889ebad5-6744-486d-aade-f9768038c205 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:09.032790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:09.047708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 287/564] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:09.047708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f540ed-293e-404f-8c3e-07020c1f31a8 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:09.047708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f540ed-293e-404f-8c3e-07020c1f31a8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:09.047708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f540ed-293e-404f-8c3e-07020c1f31a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:09.054207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e224a0-4db5-4807-ad7f-237c521d1427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:09.055009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e224a0-4db5-4807-ad7f-237c521d1427 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:09.055289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e224a0-4db5-4807-ad7f-237c521d1427 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:09.055563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e224a0-4db5-4807-ad7f-237c521d1427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:09.062252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-83f540ed-293e-404f-8c3e-07020c1f31a8 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:09.063586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-83f540ed-293e-404f-8c3e-07020c1f31a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:09.068110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e224a0-4db5-4807-ad7f-237c521d1427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:09.069887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e224a0-4db5-4807-ad7f-237c521d1427 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-56e224a0-4db5-4807-ad7f-237c521d1427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:09.089723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:09.090886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:09.092007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:09.092007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:09.092007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:09.092007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:09.098688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 278/565] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:09.098688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e224a0-4db5-4807-ad7f-237c521d1427 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:09.145176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 288/566] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:09.146061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f540ed-293e-404f-8c3e-07020c1f31a8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:09.215896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e224a0-4db5-4807-ad7f-237c521d1427 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:09.255260 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f540ed-293e-404f-8c3e-07020c1f31a8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:09.258033 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed8880-cc2a-4ead-a54b-785e864551e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:09.258533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:09.258762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed8880-cc2a-4ead-a54b-785e864551e7 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:09.259128 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed8880-cc2a-4ead-a54b-785e864551e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:09.259483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed8880-cc2a-4ead-a54b-785e864551e7 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:09.260361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed8880-cc2a-4ead-a54b-785e864551e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:09.260771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed8880-cc2a-4ead-a54b-785e864551e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:09.262383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed8880-cc2a-4ead-a54b-785e864551e7 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:09.276187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 279/567] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:09.276187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed8880-cc2a-4ead-a54b-785e864551e7 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:09.300851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4da90-0082-4775-801a-5e0b9f86dae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:09.301653 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4da90-0082-4775-801a-5e0b9f86dae6 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:09.301938 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4da90-0082-4775-801a-5e0b9f86dae6 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:09.302216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4da90-0082-4775-801a-5e0b9f86dae6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:09.310547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4da90-0082-4775-801a-5e0b9f86dae6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:09.310888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:09.312483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4da90-0082-4775-801a-5e0b9f86dae6 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-9dd4da90-0082-4775-801a-5e0b9f86dae6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:09.327030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:09.328158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:09.329180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:09.329180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:09.329180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:09.329180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:09.329180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:09.329180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:09.329180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:09.329180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:09.329180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 289/568] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:09.329745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4da90-0082-4775-801a-5e0b9f86dae6 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:09.381023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed8880-cc2a-4ead-a54b-785e864551e7 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:09.414901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab471b4-47bd-4b51-ae4a-6f286e3f599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:09.415569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab471b4-47bd-4b51-ae4a-6f286e3f599d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:09.415852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab471b4-47bd-4b51-ae4a-6f286e3f599d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:09.416174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab471b4-47bd-4b51-ae4a-6f286e3f599d None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:09.416911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab471b4-47bd-4b51-ae4a-6f286e3f599d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:09.417220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab471b4-47bd-4b51-ae4a-6f286e3f599d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:09.419120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab471b4-47bd-4b51-ae4a-6f286e3f599d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:09.431456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 280/569] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:09.431621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4da90-0082-4775-801a-5e0b9f86dae6 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:09.432499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab471b4-47bd-4b51-ae4a-6f286e3f599d None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:09.468230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c1ea9-0dc7-49c6-aa45-2379bde82ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:09.469030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c1ea9-0dc7-49c6-aa45-2379bde82ade None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:09.469314 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c1ea9-0dc7-49c6-aa45-2379bde82ade None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:09.469603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c1ea9-0dc7-49c6-aa45-2379bde82ade None admin] PATH_INFO: `/v3/roles/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:09.479650 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c1ea9-0dc7-49c6-aa45-2379bde82ade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-167c1ea9-0dc7-49c6-aa45-2379bde82ade None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:09.488092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:09.489227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:09.490081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:09.490081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:09.490081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:09.490081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:09.490081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:09.490081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:09.490081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:09.490081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:09.490081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:09.490081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:09.490081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:09.490081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:09.490081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 290/570] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:09.490826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9faa6da-1a37-4323-8dcc-e0530e6c5763 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:09.491116 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9faa6da-1a37-4323-8dcc-e0530e6c5763 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:09.491393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9faa6da-1a37-4323-8dcc-e0530e6c5763 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:09.492833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 291/571] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:09.497586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9faa6da-1a37-4323-8dcc-e0530e6c5763 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:09.522654 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:09.529842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:09.536527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab471b4-47bd-4b51-ae4a-6f286e3f599d None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:09.571514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ca773-bb24-47d4-acbd-3ea2e853e775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:09.572346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ca773-bb24-47d4-acbd-3ea2e853e775 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:09.572650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ca773-bb24-47d4-acbd-3ea2e853e775 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:09.572930 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720ca773-bb24-47d4-acbd-3ea2e853e775 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:09.582971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ca773-bb24-47d4-acbd-3ea2e853e775 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:09.584453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720ca773-bb24-47d4-acbd-3ea2e853e775 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:09.597863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9faa6da-1a37-4323-8dcc-e0530e6c5763 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-720ca773-bb24-47d4-acbd-3ea2e853e775 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:09.602443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:09.603359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:09.604280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:09.604280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:09.604280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:09.604280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:09.604280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 281/572] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:09.604557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ca773-bb24-47d4-acbd-3ea2e853e775 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:09.633950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf434a-4bed-43b3-bed7-13dd5ed5f288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:09.634643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf434a-4bed-43b3-bed7-13dd5ed5f288 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:09.634931 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf434a-4bed-43b3-bed7-13dd5ed5f288 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:09.635215 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cf434a-4bed-43b3-bed7-13dd5ed5f288 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:09.636079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf434a-4bed-43b3-bed7-13dd5ed5f288 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:09.636398 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf434a-4bed-43b3-bed7-13dd5ed5f288 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:09.637989 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cf434a-4bed-43b3-bed7-13dd5ed5f288 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:09.651188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 292/573] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:09.652088 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf434a-4bed-43b3-bed7-13dd5ed5f288 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:09.708751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720ca773-bb24-47d4-acbd-3ea2e853e775 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:09.738830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:09.744200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e0f2a-6d33-4040-88ef-7ff793a0a2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:09.745010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e0f2a-6d33-4040-88ef-7ff793a0a2d2 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:09.745296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e0f2a-6d33-4040-88ef-7ff793a0a2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:09.745575 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e0f2a-6d33-4040-88ef-7ff793a0a2d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:09.752999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e0f2a-6d33-4040-88ef-7ff793a0a2d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:09.754464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2e0f2a-6d33-4040-88ef-7ff793a0a2d2 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:09.755700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cf434a-4bed-43b3-bed7-13dd5ed5f288 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-4e2e0f2a-6d33-4040-88ef-7ff793a0a2d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:09.768259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:09.769348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:09.770570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:09.770570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:09.770570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:09.770570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:09.770570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:09.770570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:09.770570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:09.770570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:09.772067 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 282/574] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:09.773605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f92216-47b6-4501-ae55-6f5361f1fb9f None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:09.774133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f92216-47b6-4501-ae55-6f5361f1fb9f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:09.774417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f92216-47b6-4501-ae55-6f5361f1fb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:09.787203 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:09.790306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc2093-db6a-4a8c-9ba7-112aed248720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:09.790976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc2093-db6a-4a8c-9ba7-112aed248720 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:09.791255 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc2093-db6a-4a8c-9ba7-112aed248720 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:09.791530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc2093-db6a-4a8c-9ba7-112aed248720 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:09.792312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc2093-db6a-4a8c-9ba7-112aed248720 None admin] RBAC: Adding query filter params (name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:09.792613 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-d2f92216-47b6-4501-ae55-6f5361f1fb9f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:09.792723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc2093-db6a-4a8c-9ba7-112aed248720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:09.794445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d2f92216-47b6-4501-ae55-6f5361f1fb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:09.794619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc2093-db6a-4a8c-9ba7-112aed248720 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:09.802731 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 293/575] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:09.803518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc2093-db6a-4a8c-9ba7-112aed248720 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:09.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 283/576] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:09.864220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f92216-47b6-4501-ae55-6f5361f1fb9f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:09.906431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc2093-db6a-4a8c-9ba7-112aed248720 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:09.943596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54242b31-ef24-415a-b2fe-865c1dbc5e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:09.944312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54242b31-ef24-415a-b2fe-865c1dbc5e70 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:09.944598 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54242b31-ef24-415a-b2fe-865c1dbc5e70 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:09.944882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54242b31-ef24-415a-b2fe-865c1dbc5e70 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:09.945644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54242b31-ef24-415a-b2fe-865c1dbc5e70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:09.945961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54242b31-ef24-415a-b2fe-865c1dbc5e70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:09.947390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54242b31-ef24-415a-b2fe-865c1dbc5e70 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:09.960840 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 294/577] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:09.961715 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54242b31-ef24-415a-b2fe-865c1dbc5e70 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:09.973904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:09.983223 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f92216-47b6-4501-ae55-6f5361f1fb9f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:10.018475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5e8f8-276b-49e1-a4fd-1d8b4f627119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:10.019371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5e8f8-276b-49e1-a4fd-1d8b4f627119 None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:10.019649 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5e8f8-276b-49e1-a4fd-1d8b4f627119 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:10.019956 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5e8f8-276b-49e1-a4fd-1d8b4f627119 None admin] PATH_INFO: `/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/6423f54152064b909d03c0908d636993/roles/2db2bc363aa74930932c823dc291d7f3` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:10.023747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:10.039226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5e8f8-276b-49e1-a4fd-1d8b4f627119 None admin] RBAC: Authorizing `identity:create_grant(project_id=de62d2fb17f846f2bce5bef108353fd8, user_id=6423f54152064b909d03c0908d636993, role_id=2db2bc363aa74930932c823dc291d7f3)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:10.040760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5e8f8-276b-49e1-a4fd-1d8b4f627119 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:10.055709 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 284/578] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:38:09 2022] PUT /identity/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/6423f54152064b909d03c0908d636993/roles/2db2bc363aa74930932c823dc291d7f3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:38:10.056591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5e8f8-276b-49e1-a4fd-1d8b4f627119 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:10.095993 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54242b31-ef24-415a-b2fe-865c1dbc5e70 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:10.153904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334b414-404c-438e-a35b-17a825abb3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:10.155063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334b414-404c-438e-a35b-17a825abb3e9 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:10.155633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334b414-404c-438e-a35b-17a825abb3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:10.156180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8334b414-404c-438e-a35b-17a825abb3e9 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:10.157215 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334b414-404c-438e-a35b-17a825abb3e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:10.157712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334b414-404c-438e-a35b-17a825abb3e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:10.159580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8334b414-404c-438e-a35b-17a825abb3e9 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:10.176077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 295/579] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:10.177051 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334b414-404c-438e-a35b-17a825abb3e9 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:10.207459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5e8f8-276b-49e1-a4fd-1d8b4f627119 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:10.249064 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8040549-3447-4dc2-b983-c4445bd6de05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:10.250314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8040549-3447-4dc2-b983-c4445bd6de05 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:10.250903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8040549-3447-4dc2-b983-c4445bd6de05 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:10.251253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8040549-3447-4dc2-b983-c4445bd6de05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:10.254311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:10.264720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8040549-3447-4dc2-b983-c4445bd6de05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:10.266369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8040549-3447-4dc2-b983-c4445bd6de05 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-e8040549-3447-4dc2-b983-c4445bd6de05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:10.286903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:10.287998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:10.289078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:10.289078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:10.289078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:10.289078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:10.289078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 285/580] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:10.289078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8040549-3447-4dc2-b983-c4445bd6de05 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:10.292844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8334b414-404c-438e-a35b-17a825abb3e9 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:10.329768 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:10.332473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65723d6-4a3f-4ea9-afb1-d84ab2baa7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:10.333316 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65723d6-4a3f-4ea9-afb1-d84ab2baa7ee None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:10.333594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65723d6-4a3f-4ea9-afb1-d84ab2baa7ee None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:10.333866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65723d6-4a3f-4ea9-afb1-d84ab2baa7ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:10.345301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65723d6-4a3f-4ea9-afb1-d84ab2baa7ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-f65723d6-4a3f-4ea9-afb1-d84ab2baa7ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:10.353098 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:10.354339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:10.355669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:10.355669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:10.355669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:10.355669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:10.355669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:10.355669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:10.355669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:10.355669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:10.355669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:10.355669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:10.355669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:10.355669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:10.355669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 296/581] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:10.355669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65723d6-4a3f-4ea9-afb1-d84ab2baa7ee None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:10.405778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8040549-3447-4dc2-b983-c4445bd6de05 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:10.444673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c19d4b-50bc-4278-b0ac-298b28ea8d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:10.445570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c19d4b-50bc-4278-b0ac-298b28ea8d1b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:10.445970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c19d4b-50bc-4278-b0ac-298b28ea8d1b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:10.446320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c19d4b-50bc-4278-b0ac-298b28ea8d1b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:10.454608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c19d4b-50bc-4278-b0ac-298b28ea8d1b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:10.456186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c19d4b-50bc-4278-b0ac-298b28ea8d1b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:10.459473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65723d6-4a3f-4ea9-afb1-d84ab2baa7ee None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-76c19d4b-50bc-4278-b0ac-298b28ea8d1b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:10.472375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:10.473513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:10.474663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:10.474663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:10.474663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:10.474663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:10.474663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:10.474663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:10.474663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:10.474663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:10.474663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 17:38:10.474663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:10.477479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 286/582] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:10.477479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c19d4b-50bc-4278-b0ac-298b28ea8d1b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:10.494289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:10.504009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69d5e8-ca33-4c3c-988e-8606c2f53fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:10.504679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69d5e8-ca33-4c3c-988e-8606c2f53fa1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:10.504967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69d5e8-ca33-4c3c-988e-8606c2f53fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:10.505248 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69d5e8-ca33-4c3c-988e-8606c2f53fa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:10.506079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69d5e8-ca33-4c3c-988e-8606c2f53fa1 None admin] RBAC: Adding query filter params (role.id=2db2bc363aa74930932c823dc291d7f3, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=97f15306528e415baa5c69f12f22c75e) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:10.506397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69d5e8-ca33-4c3c-988e-8606c2f53fa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:10.507836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69d5e8-ca33-4c3c-988e-8606c2f53fa1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:10.514007 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 297/583] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:10 2022] GET /identity/v3/role_assignments?user.id=97f15306528e415baa5c69f12f22c75e&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=2db2bc363aa74930932c823dc291d7f3 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:10.514884 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69d5e8-ca33-4c3c-988e-8606c2f53fa1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:10.630236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:10.635996 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c19d4b-50bc-4278-b0ac-298b28ea8d1b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:10.656756 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69d5e8-ca33-4c3c-988e-8606c2f53fa1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:10.688529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b2b2c-a620-4d23-9fdd-5151e70bd098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:10.689213 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b2b2c-a620-4d23-9fdd-5151e70bd098 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:10.689497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b2b2c-a620-4d23-9fdd-5151e70bd098 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:10.689783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91b2b2c-a620-4d23-9fdd-5151e70bd098 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:10.690514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b2b2c-a620-4d23-9fdd-5151e70bd098 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:10.690827 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b2b2c-a620-4d23-9fdd-5151e70bd098 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:10.693022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91b2b2c-a620-4d23-9fdd-5151e70bd098 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:10.699676 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3193cd0-0dbc-47d8-908c-ef16ec16c6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:10.700710 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3193cd0-0dbc-47d8-908c-ef16ec16c6c1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:10.701206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3193cd0-0dbc-47d8-908c-ef16ec16c6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:10.701686 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3193cd0-0dbc-47d8-908c-ef16ec16c6c1 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:10.702636 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3193cd0-0dbc-47d8-908c-ef16ec16c6c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:10.703143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3193cd0-0dbc-47d8-908c-ef16ec16c6c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:10.704777 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3193cd0-0dbc-47d8-908c-ef16ec16c6c1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:10.705527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 287/584] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:10.706393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b2b2c-a620-4d23-9fdd-5151e70bd098 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:10.716114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 298/585] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:10.717175 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3193cd0-0dbc-47d8-908c-ef16ec16c6c1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:10.789347 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:10.850128 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3193cd0-0dbc-47d8-908c-ef16ec16c6c1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:10.859162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91b2b2c-a620-4d23-9fdd-5151e70bd098 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:10.889248 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256d4d8-ef96-41f8-8cf2-e92ca539da3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:10.890287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256d4d8-ef96-41f8-8cf2-e92ca539da3f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:10.890734 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256d4d8-ef96-41f8-8cf2-e92ca539da3f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:10.891178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f256d4d8-ef96-41f8-8cf2-e92ca539da3f None admin] PATH_INFO: `/v3/users/glance` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:10.899631 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256d4d8-ef96-41f8-8cf2-e92ca539da3f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:10.901316 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f256d4d8-ef96-41f8-8cf2-e92ca539da3f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-f256d4d8-ef96-41f8-8cf2-e92ca539da3f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:10.916722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:10.917895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:10.919655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:10.919655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:10.919655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:10.919655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:10.919655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:10.919655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:10.919655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:10.919655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:10.919655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 17:38:10.919655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:10.921871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 299/586] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:38:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:10.922792 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256d4d8-ef96-41f8-8cf2-e92ca539da3f None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:10.923032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e388b8-514d-4299-91ab-ebe8ec6e1198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:10.931590 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e388b8-514d-4299-91ab-ebe8ec6e1198 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:10.932139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e388b8-514d-4299-91ab-ebe8ec6e1198 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:10.932657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e388b8-514d-4299-91ab-ebe8ec6e1198 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:10.935208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e388b8-514d-4299-91ab-ebe8ec6e1198 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:10.935789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e388b8-514d-4299-91ab-ebe8ec6e1198 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:10.939228 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e388b8-514d-4299-91ab-ebe8ec6e1198 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:10.975570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 288/587] 10.222.0.64 () {56 vars in 1139 bytes} [Thu Jul 14 17:38:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:10.977428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e388b8-514d-4299-91ab-ebe8ec6e1198 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:11.012628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:11.034792 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f256d4d8-ef96-41f8-8cf2-e92ca539da3f None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:11.056932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:11.074419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3b58b-7728-405e-bbcf-2700cf07cc9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:11.076192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3b58b-7728-405e-bbcf-2700cf07cc9d None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:11.076694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3b58b-7728-405e-bbcf-2700cf07cc9d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:11.077150 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3b58b-7728-405e-bbcf-2700cf07cc9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:11.090835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3b58b-7728-405e-bbcf-2700cf07cc9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:11.098064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3b58b-7728-405e-bbcf-2700cf07cc9d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-25a3b58b-7728-405e-bbcf-2700cf07cc9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:11.118246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:11.119203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:11.120369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:11.120369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:11.120369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:11.120369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:11.125432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 300/588] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:11.126488 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3b58b-7728-405e-bbcf-2700cf07cc9d None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:11.187230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e388b8-514d-4299-91ab-ebe8ec6e1198 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:11.236637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67058450-9588-47b8-b7c3-f0067480ad83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:11.238550 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67058450-9588-47b8-b7c3-f0067480ad83 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:11.239346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67058450-9588-47b8-b7c3-f0067480ad83 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:11.240206 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67058450-9588-47b8-b7c3-f0067480ad83 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:11.242320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67058450-9588-47b8-b7c3-f0067480ad83 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:11.243187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67058450-9588-47b8-b7c3-f0067480ad83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:11.246022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67058450-9588-47b8-b7c3-f0067480ad83 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:11.258883 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3b58b-7728-405e-bbcf-2700cf07cc9d None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:11.276739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 289/589] 10.222.0.64 () {56 vars in 1137 bytes} [Thu Jul 14 17:38:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:11.279286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67058450-9588-47b8-b7c3-f0067480ad83 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:11.320982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c8f4f-9738-483a-a768-b72b7c63c869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:11.321817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c8f4f-9738-483a-a768-b72b7c63c869 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:11.322105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c8f4f-9738-483a-a768-b72b7c63c869 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:11.323185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c8f4f-9738-483a-a768-b72b7c63c869 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:11.338539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6c8f4f-9738-483a-a768-b72b7c63c869 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:11.342450 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6c8f4f-9738-483a-a768-b72b7c63c869 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-ad6c8f4f-9738-483a-a768-b72b7c63c869 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:11.378721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:11.379705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:11.380627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:11.380627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:11.380627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:11.380627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:11.383993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:11.385196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 301/590] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:11.386854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c8f4f-9738-483a-a768-b72b7c63c869 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:11.413376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:11.421402 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67058450-9588-47b8-b7c3-f0067480ad83 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:11.460427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f9272-3bee-48b4-bc6b-afca606b5b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:11.461545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f9272-3bee-48b4-bc6b-afca606b5b5a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:11.462108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f9272-3bee-48b4-bc6b-afca606b5b5a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:11.462594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f9272-3bee-48b4-bc6b-afca606b5b5a None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:11.463541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f9272-3bee-48b4-bc6b-afca606b5b5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:11.464099 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f9272-3bee-48b4-bc6b-afca606b5b5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:11.466367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f9272-3bee-48b4-bc6b-afca606b5b5a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:11.480186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 290/591] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:11.482159 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f9272-3bee-48b4-bc6b-afca606b5b5a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:11.513046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c8f4f-9738-483a-a768-b72b7c63c869 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:11.551797 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee80b3-f4f1-4c05-a133-4dfa1716c515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:11.552680 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee80b3-f4f1-4c05-a133-4dfa1716c515 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:11.552931 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee80b3-f4f1-4c05-a133-4dfa1716c515 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:11.553219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ee80b3-f4f1-4c05-a133-4dfa1716c515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:11.565875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee80b3-f4f1-4c05-a133-4dfa1716c515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:11.567353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ee80b3-f4f1-4c05-a133-4dfa1716c515 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-b7ee80b3-f4f1-4c05-a133-4dfa1716c515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:11.587438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:11.588434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:11.589355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:11.589355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:11.589355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:11.589355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:11.589355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 302/592] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:11.590297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee80b3-f4f1-4c05-a133-4dfa1716c515 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:11.606465 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f9272-3bee-48b4-bc6b-afca606b5b5a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:11.649033 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276abe6-6b76-49b1-a748-d210eb040b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:11.650102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276abe6-6b76-49b1-a748-d210eb040b62 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:11.650615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276abe6-6b76-49b1-a748-d210eb040b62 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:11.651097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276abe6-6b76-49b1-a748-d210eb040b62 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:11.652526 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276abe6-6b76-49b1-a748-d210eb040b62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:11.652892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276abe6-6b76-49b1-a748-d210eb040b62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:11.654300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276abe6-6b76-49b1-a748-d210eb040b62 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:11.656189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:11.675053 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 291/593] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:11.676354 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276abe6-6b76-49b1-a748-d210eb040b62 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:11.683305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:11.723834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ee80b3-f4f1-4c05-a133-4dfa1716c515 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:11.765080 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa94ec-5c67-4fbc-8557-276541a37ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:11.765905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa94ec-5c67-4fbc-8557-276541a37ae8 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:11.766189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa94ec-5c67-4fbc-8557-276541a37ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:11.766467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa94ec-5c67-4fbc-8557-276541a37ae8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:11.775424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa94ec-5c67-4fbc-8557-276541a37ae8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:11.777159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fa94ec-5c67-4fbc-8557-276541a37ae8 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-f3fa94ec-5c67-4fbc-8557-276541a37ae8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:11.793887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:11.794924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:11.795868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:11.795868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:11.795868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:11.795868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:11.795868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:11.795868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:11.795868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:11.795868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:11.795868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 17:38:11.795868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:11.800159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 303/594] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:38:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:11.801180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa94ec-5c67-4fbc-8557-276541a37ae8 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:11.823251 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276abe6-6b76-49b1-a748-d210eb040b62 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:11.868556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6e7f7-d3fa-41df-bbaf-9e0ff60417c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:11.869224 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6e7f7-d3fa-41df-bbaf-9e0ff60417c2 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:11.869510 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6e7f7-d3fa-41df-bbaf-9e0ff60417c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:11.869830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b6e7f7-d3fa-41df-bbaf-9e0ff60417c2 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:11.870593 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6e7f7-d3fa-41df-bbaf-9e0ff60417c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:11.870996 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6e7f7-d3fa-41df-bbaf-9e0ff60417c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:11.874315 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b6e7f7-d3fa-41df-bbaf-9e0ff60417c2 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:11.892795 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 292/595] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:11.894591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6e7f7-d3fa-41df-bbaf-9e0ff60417c2 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:11.922290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa94ec-5c67-4fbc-8557-276541a37ae8 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:11.959209 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:11.964673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:11.971968 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca6c10-d2f9-47c3-a791-762dc52bbe21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:11.973319 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca6c10-d2f9-47c3-a791-762dc52bbe21 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:11.973676 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca6c10-d2f9-47c3-a791-762dc52bbe21 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:11.974085 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca6c10-d2f9-47c3-a791-762dc52bbe21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:11.984633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca6c10-d2f9-47c3-a791-762dc52bbe21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:11.987119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca6c10-d2f9-47c3-a791-762dc52bbe21 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-03ca6c10-d2f9-47c3-a791-762dc52bbe21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:12.008387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:12.009366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:12.010252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:12.010252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:12.010252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:12.010252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:12.010252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:12.010252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:12.010252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:12.010252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:12.012894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 304/596] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:12.013987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca6c10-d2f9-47c3-a791-762dc52bbe21 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:12.028989 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b6e7f7-d3fa-41df-bbaf-9e0ff60417c2 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:12.068952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fba2f7-9819-4a5a-929b-2459d1be3414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:12.069643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fba2f7-9819-4a5a-929b-2459d1be3414 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:12.069930 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fba2f7-9819-4a5a-929b-2459d1be3414 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:12.070211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fba2f7-9819-4a5a-929b-2459d1be3414 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:12.071068 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fba2f7-9819-4a5a-929b-2459d1be3414 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:12.071382 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fba2f7-9819-4a5a-929b-2459d1be3414 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:12.072875 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fba2f7-9819-4a5a-929b-2459d1be3414 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:12.100159 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 293/597] 10.222.0.64 () {56 vars in 1137 bytes} [Thu Jul 14 17:38:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:12.101164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fba2f7-9819-4a5a-929b-2459d1be3414 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:12.171338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca6c10-d2f9-47c3-a791-762dc52bbe21 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:12.217398 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df731d59-e939-4b5d-870c-ee097679a534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:12.218228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df731d59-e939-4b5d-870c-ee097679a534 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:12.218516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df731d59-e939-4b5d-870c-ee097679a534 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:12.218795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df731d59-e939-4b5d-870c-ee097679a534 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:12.224357 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fba2f7-9819-4a5a-929b-2459d1be3414 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:12.226669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df731d59-e939-4b5d-870c-ee097679a534 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:12.228238 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df731d59-e939-4b5d-870c-ee097679a534 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:12.241158 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-df731d59-e939-4b5d-870c-ee097679a534 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:12.250632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:12.251836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:12.253281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:12.253281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:12.253281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:12.253281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:12.253281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:12.253281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:12.253281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:12.253281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:12.254444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 305/598] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:12.255818 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df731d59-e939-4b5d-870c-ee097679a534 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:12.272655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156f2da-941e-46fc-98a7-d5ec047bc238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:12.273750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156f2da-941e-46fc-98a7-d5ec047bc238 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:12.274117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156f2da-941e-46fc-98a7-d5ec047bc238 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:12.274666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156f2da-941e-46fc-98a7-d5ec047bc238 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:12.276009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6156f2da-941e-46fc-98a7-d5ec047bc238 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:12.276429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6156f2da-941e-46fc-98a7-d5ec047bc238 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:12.278539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6156f2da-941e-46fc-98a7-d5ec047bc238 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:12.291848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 294/599] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:12.293268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64754991-f793-4105-a4fe-21205c8c5275 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:12.293592 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64754991-f793-4105-a4fe-21205c8c5275 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:12.293873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64754991-f793-4105-a4fe-21205c8c5275 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:12.299270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 295/600] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:12.299270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64754991-f793-4105-a4fe-21205c8c5275 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:12.330251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:12.397119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df731d59-e939-4b5d-870c-ee097679a534 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:12.438877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64754991-f793-4105-a4fe-21205c8c5275 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:12.452349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568310d-cdab-4d2c-b46e-f707fb023cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:12.453150 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568310d-cdab-4d2c-b46e-f707fb023cc0 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:12.453435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568310d-cdab-4d2c-b46e-f707fb023cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:12.453718 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568310d-cdab-4d2c-b46e-f707fb023cc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:12.466230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568310d-cdab-4d2c-b46e-f707fb023cc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:12.467700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568310d-cdab-4d2c-b46e-f707fb023cc0 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:12.484252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e8cd3-101e-4de1-b69c-6ff7fb7192aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:12.484984 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e8cd3-101e-4de1-b69c-6ff7fb7192aa None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:12.485365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e8cd3-101e-4de1-b69c-6ff7fb7192aa None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:12.485766 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e8cd3-101e-4de1-b69c-6ff7fb7192aa None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:12.486673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e8cd3-101e-4de1-b69c-6ff7fb7192aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:12.487088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e8cd3-101e-4de1-b69c-6ff7fb7192aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-1568310d-cdab-4d2c-b46e-f707fb023cc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:12.487518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:12.488632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:12.495200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:12.495200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:12.495200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:12.495200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:12.495200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 306/601] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:12.495555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e8cd3-101e-4de1-b69c-6ff7fb7192aa None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:12.495642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919f7de-5d7f-421a-90d8-9289ff0e3633 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:12.495642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919f7de-5d7f-421a-90d8-9289ff0e3633 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:12.495642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4919f7de-5d7f-421a-90d8-9289ff0e3633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:12.510134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4919f7de-5d7f-421a-90d8-9289ff0e3633 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:12.516368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 296/602] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:12.516543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4919f7de-5d7f-421a-90d8-9289ff0e3633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:12.516637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e8cd3-101e-4de1-b69c-6ff7fb7192aa None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:12.552987 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:12.605373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 307/603] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:12.607180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919f7de-5d7f-421a-90d8-9289ff0e3633 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:12.650900 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:12.670385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e8cd3-101e-4de1-b69c-6ff7fb7192aa None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:12.718682 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4cf19-ea8a-4fac-b162-3d1a4a70ca50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:12.719667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4cf19-ea8a-4fac-b162-3d1a4a70ca50 None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:12.719986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4cf19-ea8a-4fac-b162-3d1a4a70ca50 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:12.720297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4cf19-ea8a-4fac-b162-3d1a4a70ca50 None admin] PATH_INFO: `/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/a5adec9924444e29b8964f7969452592/roles/2db2bc363aa74930932c823dc291d7f3` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:12.739205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4919f7de-5d7f-421a-90d8-9289ff0e3633 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:12.739995 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4cf19-ea8a-4fac-b162-3d1a4a70ca50 None admin] RBAC: Authorizing `identity:create_grant(project_id=de62d2fb17f846f2bce5bef108353fd8, user_id=a5adec9924444e29b8964f7969452592, role_id=2db2bc363aa74930932c823dc291d7f3)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:12.740882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f4cf19-ea8a-4fac-b162-3d1a4a70ca50 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:12.760021 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 297/604] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:38:12 2022] PUT /identity/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/a5adec9924444e29b8964f7969452592/roles/2db2bc363aa74930932c823dc291d7f3 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:38:12.761408 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4cf19-ea8a-4fac-b162-3d1a4a70ca50 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:12.782147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a98d2-e99a-4888-980e-41433cc8169a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:12.782833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a98d2-e99a-4888-980e-41433cc8169a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:12.783125 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a98d2-e99a-4888-980e-41433cc8169a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:12.783411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a98d2-e99a-4888-980e-41433cc8169a None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:12.784495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a98d2-e99a-4888-980e-41433cc8169a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:12.784751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a98d2-e99a-4888-980e-41433cc8169a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:12.786887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518a98d2-e99a-4888-980e-41433cc8169a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:12.802024 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 308/605] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:12.802909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a98d2-e99a-4888-980e-41433cc8169a None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:12.893712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:12.920371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f4cf19-ea8a-4fac-b162-3d1a4a70ca50 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:12.931600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:12.944437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518a98d2-e99a-4888-980e-41433cc8169a None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:12.978067 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02230a61-0f00-437f-a6fd-62704a377313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:12.979000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02230a61-0f00-437f-a6fd-62704a377313 None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:12.979468 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02230a61-0f00-437f-a6fd-62704a377313 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:12.979694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02230a61-0f00-437f-a6fd-62704a377313 None admin] PATH_INFO: `/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/b2b4bf2aa1744a46b6d2a1b5a136a0e1/roles/2db2bc363aa74930932c823dc291d7f3` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:12.987397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1932f91b-3f9b-4ee9-8bcf-cc454f887e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:12.988242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932f91b-3f9b-4ee9-8bcf-cc454f887e52 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:12.988530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932f91b-3f9b-4ee9-8bcf-cc454f887e52 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:12.988807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1932f91b-3f9b-4ee9-8bcf-cc454f887e52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:12.997633 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02230a61-0f00-437f-a6fd-62704a377313 None admin] RBAC: Authorizing `identity:create_grant(project_id=de62d2fb17f846f2bce5bef108353fd8, user_id=b2b4bf2aa1744a46b6d2a1b5a136a0e1, role_id=2db2bc363aa74930932c823dc291d7f3)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:12.999189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02230a61-0f00-437f-a6fd-62704a377313 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:13.000866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1932f91b-3f9b-4ee9-8bcf-cc454f887e52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-1932f91b-3f9b-4ee9-8bcf-cc454f887e52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:13.009072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:13.010087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:13.010950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:13.010950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:13.010950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:13.010950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:13.010950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:13.010950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:13.010950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:13.010950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:13.010950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:13.010950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:13.010950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:13.010950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:13.010950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 309/606] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:13.012736 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf353f7-4aa8-4470-b947-44139e44eb7a None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:13.012852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf353f7-4aa8-4470-b947-44139e44eb7a None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:13.013330 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf353f7-4aa8-4470-b947-44139e44eb7a None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:13.016704 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 310/607] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:13.016704 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf353f7-4aa8-4470-b947-44139e44eb7a None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:13.036045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 298/608] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:38:12 2022] PUT /identity/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/b2b4bf2aa1744a46b6d2a1b5a136a0e1/roles/2db2bc363aa74930932c823dc291d7f3 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:38:13.037471 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02230a61-0f00-437f-a6fd-62704a377313 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:13.179758 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf353f7-4aa8-4470-b947-44139e44eb7a None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:13.202192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02230a61-0f00-437f-a6fd-62704a377313 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:13.226957 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06c00ac-4da3-464a-a3b4-c6cd4a8f1f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:13.228139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c00ac-4da3-464a-a3b4-c6cd4a8f1f8a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:13.228638 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c00ac-4da3-464a-a3b4-c6cd4a8f1f8a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:13.229617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06c00ac-4da3-464a-a3b4-c6cd4a8f1f8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:13.239544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c00ac-4da3-464a-a3b4-c6cd4a8f1f8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:13.240751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06c00ac-4da3-464a-a3b4-c6cd4a8f1f8a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:13.243381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:13.249490 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68230375-889d-485a-88e5-ffe25b37813c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:13.250245 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68230375-889d-485a-88e5-ffe25b37813c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:13.251112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68230375-889d-485a-88e5-ffe25b37813c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:13.252843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68230375-889d-485a-88e5-ffe25b37813c None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:13.253692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68230375-889d-485a-88e5-ffe25b37813c None admin] RBAC: Adding query filter params (name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:13.254062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68230375-889d-485a-88e5-ffe25b37813c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:13.255684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68230375-889d-485a-88e5-ffe25b37813c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-c06c00ac-4da3-464a-a3b4-c6cd4a8f1f8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:13.263432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:13.264497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:13.265390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:13.265390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:13.265390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:13.265390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:13.265390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:13.265390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:13.265390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:13.265390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:13.265390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 311/609] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:13.269116 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc262f-36f2-44be-8f05-11cad5d4bb89 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:13.269116 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc262f-36f2-44be-8f05-11cad5d4bb89 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:13.269116 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc262f-36f2-44be-8f05-11cad5d4bb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:13.274633 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 299/610] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:13.275885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68230375-889d-485a-88e5-ffe25b37813c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:13.310078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-13cc262f-36f2-44be-8f05-11cad5d4bb89 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:13.310078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-13cc262f-36f2-44be-8f05-11cad5d4bb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:13.318254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:13.462514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68230375-889d-485a-88e5-ffe25b37813c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:13.529508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b724d-c270-4467-8689-4656817620a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:13.530098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b724d-c270-4467-8689-4656817620a9 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:13.530384 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b724d-c270-4467-8689-4656817620a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:13.535951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b724d-c270-4467-8689-4656817620a9 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:13.535951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b724d-c270-4467-8689-4656817620a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:13.535951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b724d-c270-4467-8689-4656817620a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:13.536235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 312/611] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:13.537154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc262f-36f2-44be-8f05-11cad5d4bb89 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:13.539132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b724d-c270-4467-8689-4656817620a9 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:13.591960 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 300/612] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:13.591960 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b724d-c270-4467-8689-4656817620a9 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:13.710084 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:13.730881 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc262f-36f2-44be-8f05-11cad5d4bb89 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:13.761380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b724d-c270-4467-8689-4656817620a9 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:13.777570 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e3167-ad32-4c6b-8558-15ffa139c464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:13.780343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e3167-ad32-4c6b-8558-15ffa139c464 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:13.780789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e3167-ad32-4c6b-8558-15ffa139c464 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:13.781225 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e3167-ad32-4c6b-8558-15ffa139c464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:13.792972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:13.796450 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e3167-ad32-4c6b-8558-15ffa139c464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:13.798697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4e3167-ad32-4c6b-8558-15ffa139c464 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:13.807650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c341c-5ce6-4c3c-8073-0fed1d40fb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:13.808642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c341c-5ce6-4c3c-8073-0fed1d40fb2a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:13.809067 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c341c-5ce6-4c3c-8073-0fed1d40fb2a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:13.810499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c341c-5ce6-4c3c-8073-0fed1d40fb2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:13.822610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c341c-5ce6-4c3c-8073-0fed1d40fb2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-ce4e3167-ad32-4c6b-8558-15ffa139c464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:13.823892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:13.824872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:13.825765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:13.825765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:13.825765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:13.825765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-523c341c-5ce6-4c3c-8073-0fed1d40fb2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:13.831917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:13.832870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:13.833703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:13.833703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:13.833703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:13.833703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:13.833703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:13.833703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:13.833703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:13.833703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:13.833703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:13.833703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:13.833703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:13.833703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:13.833703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 301/613] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:13.834380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c341c-5ce6-4c3c-8073-0fed1d40fb2a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:13.835116 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 313/614] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:13.836321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e3167-ad32-4c6b-8558-15ffa139c464 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:13.970768 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c341c-5ce6-4c3c-8073-0fed1d40fb2a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:13.982398 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e3167-ad32-4c6b-8558-15ffa139c464 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:13.988474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:14.014687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82048a35-efe4-4a69-b8e3-ced27762cbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:14.015982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82048a35-efe4-4a69-b8e3-ced27762cbbc None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:14.016450 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82048a35-efe4-4a69-b8e3-ced27762cbbc None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:14.016939 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82048a35-efe4-4a69-b8e3-ced27762cbbc None admin] PATH_INFO: `/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/e49fad7db3c643b99e9071d9ca5f0196/roles/2db2bc363aa74930932c823dc291d7f3` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:14.020488 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064c7b8-26f5-4d26-9549-28ba44516d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:14.021232 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064c7b8-26f5-4d26-9549-28ba44516d90 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:14.022308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064c7b8-26f5-4d26-9549-28ba44516d90 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:14.022308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064c7b8-26f5-4d26-9549-28ba44516d90 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:14.022717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064c7b8-26f5-4d26-9549-28ba44516d90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:14.023095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064c7b8-26f5-4d26-9549-28ba44516d90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:14.024592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064c7b8-26f5-4d26-9549-28ba44516d90 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:14.035912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82048a35-efe4-4a69-b8e3-ced27762cbbc None admin] RBAC: Authorizing `identity:create_grant(project_id=de62d2fb17f846f2bce5bef108353fd8, user_id=e49fad7db3c643b99e9071d9ca5f0196, role_id=2db2bc363aa74930932c823dc291d7f3)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:14.041970 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 314/615] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:14.041970 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064c7b8-26f5-4d26-9549-28ba44516d90 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:14.042810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82048a35-efe4-4a69-b8e3-ced27762cbbc None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:14.067170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 302/616] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:38:13 2022] PUT /identity/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/e49fad7db3c643b99e9071d9ca5f0196/roles/2db2bc363aa74930932c823dc291d7f3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:38:14.068809 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82048a35-efe4-4a69-b8e3-ced27762cbbc None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:14.101807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:14.158597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064c7b8-26f5-4d26-9549-28ba44516d90 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:14.199397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6146d6-f35e-4570-a1df-313b2365ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:14.200797 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6146d6-f35e-4570-a1df-313b2365ad45 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:14.201153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6146d6-f35e-4570-a1df-313b2365ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:14.201508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6146d6-f35e-4570-a1df-313b2365ad45 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:14.202404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6146d6-f35e-4570-a1df-313b2365ad45 None admin] RBAC: Adding query filter params (name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:14.202821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6146d6-f35e-4570-a1df-313b2365ad45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:14.204427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6146d6-f35e-4570-a1df-313b2365ad45 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:14.214435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 315/617] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:14.222659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6146d6-f35e-4570-a1df-313b2365ad45 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:14.284053 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82048a35-efe4-4a69-b8e3-ced27762cbbc None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:14.346534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5429613-9712-41f7-b005-41f7b248ae65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:14.347885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5429613-9712-41f7-b005-41f7b248ae65 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:14.348400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5429613-9712-41f7-b005-41f7b248ae65 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:14.348903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5429613-9712-41f7-b005-41f7b248ae65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:14.378976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5429613-9712-41f7-b005-41f7b248ae65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:14.381760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5429613-9712-41f7-b005-41f7b248ae65 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-c5429613-9712-41f7-b005-41f7b248ae65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:14.415541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:14.416628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:14.418115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:14.418115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:14.418115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:14.418115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:14.418115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:14.418115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:14.418115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:14.418115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:14.418115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:38:14.418115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:14.418115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 303/618] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Jul 14 17:38:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:14.425362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5429613-9712-41f7-b005-41f7b248ae65 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:14.477155 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:14.567821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6146d6-f35e-4570-a1df-313b2365ad45 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:14.597349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:14.636205 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5429613-9712-41f7-b005-41f7b248ae65 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:14.661719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb4bea-9998-489e-8c96-11a598868451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:14.662542 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb4bea-9998-489e-8c96-11a598868451 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:14.662814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb4bea-9998-489e-8c96-11a598868451 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:14.663095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb4bea-9998-489e-8c96-11a598868451 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:14.689480 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb4bea-9998-489e-8c96-11a598868451 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:14.690956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb4bea-9998-489e-8c96-11a598868451 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:14.708879 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94fc17-ccdd-4d10-93f5-b003d2e8b23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:14.710630 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94fc17-ccdd-4d10-93f5-b003d2e8b23c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:14.711425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94fc17-ccdd-4d10-93f5-b003d2e8b23c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:14.712252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94fc17-ccdd-4d10-93f5-b003d2e8b23c None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:14.714220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94fc17-ccdd-4d10-93f5-b003d2e8b23c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:14.715101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94fc17-ccdd-4d10-93f5-b003d2e8b23c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:14.718266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94fc17-ccdd-4d10-93f5-b003d2e8b23c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-85bb4bea-9998-489e-8c96-11a598868451 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:14.722481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:14.724474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:14.726277 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:14.726277 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:14.726277 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:14.726277 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:14.728290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 316/619] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 506 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:14.745587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb4bea-9998-489e-8c96-11a598868451 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:14.763086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 304/620] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Jul 14 17:38:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:14.771725 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94fc17-ccdd-4d10-93f5-b003d2e8b23c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:14.891799 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:14.913419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94fc17-ccdd-4d10-93f5-b003d2e8b23c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:14.954843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474aec1-9018-4268-aa52-74c7b6cfac65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:14.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474aec1-9018-4268-aa52-74c7b6cfac65 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:14.955827 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474aec1-9018-4268-aa52-74c7b6cfac65 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:14.956155 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474aec1-9018-4268-aa52-74c7b6cfac65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:15.032456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474aec1-9018-4268-aa52-74c7b6cfac65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:15.034086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474aec1-9018-4268-aa52-74c7b6cfac65 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:15.053927 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb4bea-9998-489e-8c96-11a598868451 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:15.129142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:15.143211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 305/621] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 17:38:15.148645 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474aec1-9018-4268-aa52-74c7b6cfac65 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:15.174913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3b7d5-b078-4e46-bc61-a1b8b5b2e36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:15.176561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3b7d5-b078-4e46-bc61-a1b8b5b2e36d None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:15.176919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3b7d5-b078-4e46-bc61-a1b8b5b2e36d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:15.177362 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3b7d5-b078-4e46-bc61-a1b8b5b2e36d None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:15.178524 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3b7d5-b078-4e46-bc61-a1b8b5b2e36d None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:15.178914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3b7d5-b078-4e46-bc61-a1b8b5b2e36d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:15.181111 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e3b7d5-b078-4e46-bc61-a1b8b5b2e36d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:15.215307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 317/622] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 471 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:15.220935 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3b7d5-b078-4e46-bc61-a1b8b5b2e36d None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:15.256543 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474aec1-9018-4268-aa52-74c7b6cfac65 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:15.294350 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ecd4e-e739-4133-8103-bf37dfb09798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:15.295209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ecd4e-e739-4133-8103-bf37dfb09798 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:15.295591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ecd4e-e739-4133-8103-bf37dfb09798 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:15.295884 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ecd4e-e739-4133-8103-bf37dfb09798 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:15.307438 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ecd4e-e739-4133-8103-bf37dfb09798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:15.308962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ecd4e-e739-4133-8103-bf37dfb09798 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:15.321638 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-a28ecd4e-e739-4133-8103-bf37dfb09798 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:15.330631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:15.331603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:15.332571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:15.332571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:15.332571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:15.332571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:15.336390 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 306/623] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:15.337936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ecd4e-e739-4133-8103-bf37dfb09798 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:15.386813 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:15.433334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e3b7d5-b078-4e46-bc61-a1b8b5b2e36d None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:15.473742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ecd4e-e739-4133-8103-bf37dfb09798 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:15.480045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a03e1d-8e1d-4784-91e4-138857e42356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:15.480880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a03e1d-8e1d-4784-91e4-138857e42356 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:15.481767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a03e1d-8e1d-4784-91e4-138857e42356 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:15.482049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a03e1d-8e1d-4784-91e4-138857e42356 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:15.490799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a03e1d-8e1d-4784-91e4-138857e42356 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:15.492425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a03e1d-8e1d-4784-91e4-138857e42356 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-85a03e1d-8e1d-4784-91e4-138857e42356 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:15.510557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:15.511578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:15.512694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:15.512694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:15.512694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:15.512694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:15.512694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:15.512694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:15.512694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:15.512694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:15.512694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 17:38:15.512694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:15.512694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 318/624] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:38:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:15.513795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4b67c-c264-4107-b43c-4ca973fe9b5b None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:15.516730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4b67c-c264-4107-b43c-4ca973fe9b5b None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:15.516730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce4b67c-c264-4107-b43c-4ca973fe9b5b None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:15.517978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 319/625] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:15.519025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4b67c-c264-4107-b43c-4ca973fe9b5b None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:15.524584 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf90d6d-fc67-4b15-a489-95ef2a30c4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:15.525610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf90d6d-fc67-4b15-a489-95ef2a30c4c4 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:15.526018 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf90d6d-fc67-4b15-a489-95ef2a30c4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:15.526298 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf90d6d-fc67-4b15-a489-95ef2a30c4c4 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:15.527049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf90d6d-fc67-4b15-a489-95ef2a30c4c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:15.527355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf90d6d-fc67-4b15-a489-95ef2a30c4c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:15.528868 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf90d6d-fc67-4b15-a489-95ef2a30c4c4 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:15.543326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 307/626] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:15.549798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762aa8d-bf9c-4c15-9eda-514f340f5e81 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:15.550122 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762aa8d-bf9c-4c15-9eda-514f340f5e81 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:15.550406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f762aa8d-bf9c-4c15-9eda-514f340f5e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:15.572196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f762aa8d-bf9c-4c15-9eda-514f340f5e81 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:15.574156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f762aa8d-bf9c-4c15-9eda-514f340f5e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:15.617994 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:15.639688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce4b67c-c264-4107-b43c-4ca973fe9b5b None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:15.686906 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ae099-3fc0-4cec-9af6-d81674ca8c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:15.687843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ae099-3fc0-4cec-9af6-d81674ca8c16 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:15.688803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ae099-3fc0-4cec-9af6-d81674ca8c16 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:15.689234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ae099-3fc0-4cec-9af6-d81674ca8c16 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:15.690580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ae099-3fc0-4cec-9af6-d81674ca8c16 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:15.691055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ae099-3fc0-4cec-9af6-d81674ca8c16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:15.692917 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ae099-3fc0-4cec-9af6-d81674ca8c16 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:15.700583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 308/627] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:15.702105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762aa8d-bf9c-4c15-9eda-514f340f5e81 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:15.725729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 320/628] 10.222.0.64 () {56 vars in 1135 bytes} [Thu Jul 14 17:38:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:15.727567 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d2cba-1842-4274-b8c0-10c3ab346d66 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:15.728046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d2cba-1842-4274-b8c0-10c3ab346d66 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:15.728551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d2cba-1842-4274-b8c0-10c3ab346d66 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:15.734779 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 321/629] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:15.736033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d2cba-1842-4274-b8c0-10c3ab346d66 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:15.747819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:15.844127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f762aa8d-bf9c-4c15-9eda-514f340f5e81 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:15.888495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1d2cba-1842-4274-b8c0-10c3ab346d66 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:15.890740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802fdf6-121a-4fb3-9e62-09e43800b21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:15.891730 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802fdf6-121a-4fb3-9e62-09e43800b21e None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:15.892050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802fdf6-121a-4fb3-9e62-09e43800b21e None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:15.893148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802fdf6-121a-4fb3-9e62-09e43800b21e None admin] PATH_INFO: `/v3/projects/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:15.900266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802fdf6-121a-4fb3-9e62-09e43800b21e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:15.902167 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802fdf6-121a-4fb3-9e62-09e43800b21e None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-d802fdf6-121a-4fb3-9e62-09e43800b21e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:15.917316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:15.919903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:15.921788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:15.921788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:15.921788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:15.921788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:15.921788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:15.921788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:15.921788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:15.921788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:15.921788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 309/630] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:15.921788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802fdf6-121a-4fb3-9e62-09e43800b21e None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:15.949105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6c027-1b1b-49bd-97e9-3959fb985745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:15.950503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6c027-1b1b-49bd-97e9-3959fb985745 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:15.950797 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6c027-1b1b-49bd-97e9-3959fb985745 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:15.951074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6c027-1b1b-49bd-97e9-3959fb985745 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:15.962525 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6c027-1b1b-49bd-97e9-3959fb985745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-21a6c027-1b1b-49bd-97e9-3959fb985745 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:15.970964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:15.972936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:15.974632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:15.974632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:15.974632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:15.974632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:15.974632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:15.974632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:15.974632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:15.974632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:15.974632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:15.974632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:15.974632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:15.974632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:15.977344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 322/631] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:15.978713 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b17bd-b92c-4d6a-b93e-811eb9107e17 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:15.978998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b17bd-b92c-4d6a-b93e-811eb9107e17 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:15.979262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b17bd-b92c-4d6a-b93e-811eb9107e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:15.987282 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:15.999058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-e83b17bd-b92c-4d6a-b93e-811eb9107e17 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:16.000309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e83b17bd-b92c-4d6a-b93e-811eb9107e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:16.042030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:16.076697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802fdf6-121a-4fb3-9e62-09e43800b21e None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:16.081170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 323/632] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:16.082470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b17bd-b92c-4d6a-b93e-811eb9107e17 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:16.147205 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a635a5-9a33-4925-aef8-ea80df4ae555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:16.148744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a635a5-9a33-4925-aef8-ea80df4ae555 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:16.149591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a635a5-9a33-4925-aef8-ea80df4ae555 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:16.149827 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a635a5-9a33-4925-aef8-ea80df4ae555 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:16.171576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a635a5-9a33-4925-aef8-ea80df4ae555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:16.174371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a635a5-9a33-4925-aef8-ea80df4ae555 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:16.199042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b17bd-b92c-4d6a-b93e-811eb9107e17 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-c3a635a5-9a33-4925-aef8-ea80df4ae555 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:16.208625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:16.209522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:16.210409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:16.210409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:16.210409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:16.210409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:16.210905 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 310/633] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:16.212236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a635a5-9a33-4925-aef8-ea80df4ae555 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:16.240786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07151d6b-67d0-4faf-a026-dd6f1a058451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:16.241882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07151d6b-67d0-4faf-a026-dd6f1a058451 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:16.242363 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07151d6b-67d0-4faf-a026-dd6f1a058451 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:16.242798 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07151d6b-67d0-4faf-a026-dd6f1a058451 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:16.243749 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07151d6b-67d0-4faf-a026-dd6f1a058451 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:16.244276 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07151d6b-67d0-4faf-a026-dd6f1a058451 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:16.245879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07151d6b-67d0-4faf-a026-dd6f1a058451 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:16.248066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:16.265409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 324/634] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:16.265409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07151d6b-67d0-4faf-a026-dd6f1a058451 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:16.356887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a635a5-9a33-4925-aef8-ea80df4ae555 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:16.385543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07151d6b-67d0-4faf-a026-dd6f1a058451 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:16.414008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4355ae-3caf-491f-9f6d-247911e8d198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:16.414721 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4355ae-3caf-491f-9f6d-247911e8d198 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:16.415007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4355ae-3caf-491f-9f6d-247911e8d198 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:16.415288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4355ae-3caf-491f-9f6d-247911e8d198 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:16.416098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4355ae-3caf-491f-9f6d-247911e8d198 None admin] RBAC: Adding query filter params (name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:16.416412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4355ae-3caf-491f-9f6d-247911e8d198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:16.417842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4355ae-3caf-491f-9f6d-247911e8d198 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:16.424349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:16.433489 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71455566-9530-4291-8c95-8ef675a06b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:16.435315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71455566-9530-4291-8c95-8ef675a06b0b None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:16.435797 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71455566-9530-4291-8c95-8ef675a06b0b None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:16.436316 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71455566-9530-4291-8c95-8ef675a06b0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:16.440322 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 311/635] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:16.442129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4355ae-3caf-491f-9f6d-247911e8d198 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:16.453981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71455566-9530-4291-8c95-8ef675a06b0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-71455566-9530-4291-8c95-8ef675a06b0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:16.462930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:16.464202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:16.465071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:16.465071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:16.465071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:16.465071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:16.465071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:16.465071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:16.465071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:16.465071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:16.465071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:16.465071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:16.465071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:16.465071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:16.468469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 325/636] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:16.470195 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71455566-9530-4291-8c95-8ef675a06b0b None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:16.531664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:16.556379 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4355ae-3caf-491f-9f6d-247911e8d198 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:16.580554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71455566-9530-4291-8c95-8ef675a06b0b None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:16.594194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9106dc3-6d0d-4130-999e-8e0b2ee048a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:16.594883 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9106dc3-6d0d-4130-999e-8e0b2ee048a7 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:16.595169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9106dc3-6d0d-4130-999e-8e0b2ee048a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:16.595447 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9106dc3-6d0d-4130-999e-8e0b2ee048a7 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:16.596210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9106dc3-6d0d-4130-999e-8e0b2ee048a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:16.596523 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9106dc3-6d0d-4130-999e-8e0b2ee048a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:16.597900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9106dc3-6d0d-4130-999e-8e0b2ee048a7 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:16.611434 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 312/637] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:16.612455 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9106dc3-6d0d-4130-999e-8e0b2ee048a7 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:16.621821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf25ab-30f8-49b7-9e04-6761e87c7600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:16.622536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf25ab-30f8-49b7-9e04-6761e87c7600 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:16.622822 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf25ab-30f8-49b7-9e04-6761e87c7600 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:16.623102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf25ab-30f8-49b7-9e04-6761e87c7600 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:16.623971 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf25ab-30f8-49b7-9e04-6761e87c7600 None admin] RBAC: Adding query filter params (role.id=2db2bc363aa74930932c823dc291d7f3, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=6423f54152064b909d03c0908d636993) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:16.624293 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf25ab-30f8-49b7-9e04-6761e87c7600 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:16.625738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf25ab-30f8-49b7-9e04-6761e87c7600 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:16.639973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 326/638] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:16 2022] GET /identity/v3/role_assignments?user.id=6423f54152064b909d03c0908d636993&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=2db2bc363aa74930932c823dc291d7f3 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:16.639973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf25ab-30f8-49b7-9e04-6761e87c7600 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:16.701676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:16.753764 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9106dc3-6d0d-4130-999e-8e0b2ee048a7 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:16.794208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c3ba94-129d-4bf4-a193-847f1e4b371a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:16.795013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf25ab-30f8-49b7-9e04-6761e87c7600 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:16.795125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3ba94-129d-4bf4-a193-847f1e4b371a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:16.795298 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3ba94-129d-4bf4-a193-847f1e4b371a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:16.795573 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c3ba94-129d-4bf4-a193-847f1e4b371a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:16.807676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c3ba94-129d-4bf4-a193-847f1e4b371a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:16.808643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c3ba94-129d-4bf4-a193-847f1e4b371a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:16.819910 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-59c3ba94-129d-4bf4-a193-847f1e4b371a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:16.831738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:16.833511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:16.835264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:16.835264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:16.835264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:16.835264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:16.835264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 313/639] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:16.835264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c3ba94-129d-4bf4-a193-847f1e4b371a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:16.846809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb358a-9086-4814-950f-768e42377026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:16.847576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb358a-9086-4814-950f-768e42377026 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:16.847845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb358a-9086-4814-950f-768e42377026 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:16.848147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb358a-9086-4814-950f-768e42377026 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:16.848918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb358a-9086-4814-950f-768e42377026 None admin] RBAC: Adding query filter params (name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:16.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb358a-9086-4814-950f-768e42377026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:16.850602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb358a-9086-4814-950f-768e42377026 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:16.860304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 327/640] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:16.861713 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735dec3-b9d8-4804-aebe-2507c5aa3a24 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:16.862005 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735dec3-b9d8-4804-aebe-2507c5aa3a24 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:16.862283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735dec3-b9d8-4804-aebe-2507c5aa3a24 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:16.863816 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 328/641] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:16.864590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735dec3-b9d8-4804-aebe-2507c5aa3a24 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:16.991700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c3ba94-129d-4bf4-a193-847f1e4b371a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:16.995184 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735dec3-b9d8-4804-aebe-2507c5aa3a24 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:17.004634 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:17.030559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3452ba1a-51f6-4178-b880-75450e7a84f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:17.031657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452ba1a-51f6-4178-b880-75450e7a84f0 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:17.031965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452ba1a-51f6-4178-b880-75450e7a84f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:17.032251 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452ba1a-51f6-4178-b880-75450e7a84f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:17.034703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f30a6-fe1b-41c7-be30-6653f325b8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:17.035687 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f30a6-fe1b-41c7-be30-6653f325b8dd None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:17.036026 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f30a6-fe1b-41c7-be30-6653f325b8dd None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:17.036307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5f30a6-fe1b-41c7-be30-6653f325b8dd None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:17.037038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f30a6-fe1b-41c7-be30-6653f325b8dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:17.037343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f30a6-fe1b-41c7-be30-6653f325b8dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:17.039044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5f30a6-fe1b-41c7-be30-6653f325b8dd None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:17.039338 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452ba1a-51f6-4178-b880-75450e7a84f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:17.040802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3452ba1a-51f6-4178-b880-75450e7a84f0 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:17.057278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 329/642] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:17.057278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f30a6-fe1b-41c7-be30-6653f325b8dd None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-3452ba1a-51f6-4178-b880-75450e7a84f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:17.057558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:17.058761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:17.060300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:17.060300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:17.060300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:17.060300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:17.060300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:17.060300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:17.060300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:17.060300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:17.060300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 314/643] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:17.060300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51cf5e-1553-4258-a3e0-b8cbdd0ace1a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:17.060300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51cf5e-1553-4258-a3e0-b8cbdd0ace1a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:17.060922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51cf5e-1553-4258-a3e0-b8cbdd0ace1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:17.077283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-be51cf5e-1553-4258-a3e0-b8cbdd0ace1a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:17.078551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-be51cf5e-1553-4258-a3e0-b8cbdd0ace1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:17.116577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:17.156231 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 315/644] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:17.156953 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51cf5e-1553-4258-a3e0-b8cbdd0ace1a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:17.192093 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5f30a6-fe1b-41c7-be30-6653f325b8dd None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:17.233864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbb402-d920-4c20-86f6-7ea548b6a6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:17.234712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbb402-d920-4c20-86f6-7ea548b6a6bc None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:17.234998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbb402-d920-4c20-86f6-7ea548b6a6bc None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:17.235274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cbb402-d920-4c20-86f6-7ea548b6a6bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:17.248160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbb402-d920-4c20-86f6-7ea548b6a6bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:17.249683 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cbb402-d920-4c20-86f6-7ea548b6a6bc None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-12cbb402-d920-4c20-86f6-7ea548b6a6bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:17.269919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:17.270867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:17.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:17.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:17.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:17.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:17.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 330/645] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:17.272484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbb402-d920-4c20-86f6-7ea548b6a6bc None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:17.280746 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:17.283836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51cf5e-1553-4258-a3e0-b8cbdd0ace1a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:17.325670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64373b57-4a6f-4d67-bbcd-d26b67c98d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:17.326530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64373b57-4a6f-4d67-bbcd-d26b67c98d78 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:17.326815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64373b57-4a6f-4d67-bbcd-d26b67c98d78 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:17.327092 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64373b57-4a6f-4d67-bbcd-d26b67c98d78 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:17.337118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64373b57-4a6f-4d67-bbcd-d26b67c98d78 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:17.338984 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64373b57-4a6f-4d67-bbcd-d26b67c98d78 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-64373b57-4a6f-4d67-bbcd-d26b67c98d78 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:17.358111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:17.359388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:17.360627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:17.360627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:17.360627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:17.360627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:17.360627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:17.360627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:17.360627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:17.360627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:17.360627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 17:38:17.360627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:17.362111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 316/646] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:17.362891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64373b57-4a6f-4d67-bbcd-d26b67c98d78 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:17.385182 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:17.396043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cbb402-d920-4c20-86f6-7ea548b6a6bc None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:17.435050 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da648587-11de-4beb-b6d8-8473d4002697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:17.435817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da648587-11de-4beb-b6d8-8473d4002697 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:17.436185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da648587-11de-4beb-b6d8-8473d4002697 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:17.436523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da648587-11de-4beb-b6d8-8473d4002697 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:17.437401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da648587-11de-4beb-b6d8-8473d4002697 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:17.437860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da648587-11de-4beb-b6d8-8473d4002697 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:17.439297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da648587-11de-4beb-b6d8-8473d4002697 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:17.457617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 331/647] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:17.457617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da648587-11de-4beb-b6d8-8473d4002697 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:17.491058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64373b57-4a6f-4d67-bbcd-d26b67c98d78 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:17.530232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4f98e-d6d9-4a43-876e-d44e83d1d015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:17.530957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4f98e-d6d9-4a43-876e-d44e83d1d015 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:17.531240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4f98e-d6d9-4a43-876e-d44e83d1d015 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:17.531517 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4f98e-d6d9-4a43-876e-d44e83d1d015 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:17.542578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4f98e-d6d9-4a43-876e-d44e83d1d015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-86c4f98e-d6d9-4a43-876e-d44e83d1d015 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:17.550566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:17.551545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:17.552425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:17.552425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:17.552425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:17.552425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:17.552425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:17.552425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:17.552425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:17.552425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:17.552425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:17.552425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:17.552425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:17.552425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:17.552425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 317/648] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:17.553118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4f98e-d6d9-4a43-876e-d44e83d1d015 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:17.573683 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da648587-11de-4beb-b6d8-8473d4002697 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:17.583647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:17.613577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d3820-ff2d-4cdc-88c6-169c000e3bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:17.614256 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d3820-ff2d-4cdc-88c6-169c000e3bcf None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:17.614540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d3820-ff2d-4cdc-88c6-169c000e3bcf None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:17.614816 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d3820-ff2d-4cdc-88c6-169c000e3bcf None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:17.615551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d3820-ff2d-4cdc-88c6-169c000e3bcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:17.615861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d3820-ff2d-4cdc-88c6-169c000e3bcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:17.617272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d3820-ff2d-4cdc-88c6-169c000e3bcf None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:17.631599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 332/649] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:17.632724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d3820-ff2d-4cdc-88c6-169c000e3bcf None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:17.649599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:17.693762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4f98e-d6d9-4a43-876e-d44e83d1d015 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:17.732962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30361dc-bd18-4679-84b0-80b232054f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:17.734083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30361dc-bd18-4679-84b0-80b232054f1b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:17.734594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30361dc-bd18-4679-84b0-80b232054f1b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:17.734971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30361dc-bd18-4679-84b0-80b232054f1b None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:17.735946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30361dc-bd18-4679-84b0-80b232054f1b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:17.736356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30361dc-bd18-4679-84b0-80b232054f1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:17.737962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30361dc-bd18-4679-84b0-80b232054f1b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:17.741828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d3820-ff2d-4cdc-88c6-169c000e3bcf None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:17.762602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 318/650] 10.222.0.64 () {56 vars in 1139 bytes} [Thu Jul 14 17:38:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:17.763546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30361dc-bd18-4679-84b0-80b232054f1b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:17.780675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4cca7-a717-4b32-abae-9287a4458ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:17.781480 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4cca7-a717-4b32-abae-9287a4458ab9 None admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:17.781764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4cca7-a717-4b32-abae-9287a4458ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:17.782073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4cca7-a717-4b32-abae-9287a4458ab9 None admin] PATH_INFO: `/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/97f15306528e415baa5c69f12f22c75e/roles/2db2bc363aa74930932c823dc291d7f3` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:17.798778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b4cca7-a717-4b32-abae-9287a4458ab9 None admin] RBAC: Authorizing `identity:create_grant(project_id=de62d2fb17f846f2bce5bef108353fd8, user_id=97f15306528e415baa5c69f12f22c75e, role_id=2db2bc363aa74930932c823dc291d7f3)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:17.800403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b4cca7-a717-4b32-abae-9287a4458ab9 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:17.823966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 333/651] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:38:17 2022] PUT /identity/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/97f15306528e415baa5c69f12f22c75e/roles/2db2bc363aa74930932c823dc291d7f3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:38:17.823966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4cca7-a717-4b32-abae-9287a4458ab9 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:17.872831 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:17.904577 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30361dc-bd18-4679-84b0-80b232054f1b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:17.908748 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:17.948855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b908a4-30af-44c5-b058-69bcdd2b6716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:17.949610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b908a4-30af-44c5-b058-69bcdd2b6716 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:17.950558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b908a4-30af-44c5-b058-69bcdd2b6716 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:17.950842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b908a4-30af-44c5-b058-69bcdd2b6716 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:17.951622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b908a4-30af-44c5-b058-69bcdd2b6716 None admin] RBAC: Adding query filter params (name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:17.951958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b908a4-30af-44c5-b058-69bcdd2b6716 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:17.953375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b908a4-30af-44c5-b058-69bcdd2b6716 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:17.968680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 319/652] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:17.969683 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b908a4-30af-44c5-b058-69bcdd2b6716 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:17.970420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4cca7-a717-4b32-abae-9287a4458ab9 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:18.015908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06129c3-7209-40c7-af37-a060b2a017d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:18.017089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06129c3-7209-40c7-af37-a060b2a017d1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:18.017589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06129c3-7209-40c7-af37-a060b2a017d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:18.018095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06129c3-7209-40c7-af37-a060b2a017d1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:18.028096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06129c3-7209-40c7-af37-a060b2a017d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:18.031974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06129c3-7209-40c7-af37-a060b2a017d1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-b06129c3-7209-40c7-af37-a060b2a017d1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:18.050778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:18.052105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:18.053384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:18.053384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:18.053384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:18.053384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:18.053384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:18.053384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:18.053384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:18.053384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:18.053384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 17:38:18.053384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:18.055103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 334/653] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:38:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:18.059948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06129c3-7209-40c7-af37-a060b2a017d1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:18.106519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b908a4-30af-44c5-b058-69bcdd2b6716 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:18.159679 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1ae9d-d867-46d9-abd9-89ac28cf6766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:18.161037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1ae9d-d867-46d9-abd9-89ac28cf6766 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:18.161920 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1ae9d-d867-46d9-abd9-89ac28cf6766 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:18.162536 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1ae9d-d867-46d9-abd9-89ac28cf6766 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:18.180501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:18.203852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06129c3-7209-40c7-af37-a060b2a017d1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:18.234100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:18.250676 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6c643-69b7-4edf-9bfa-091ba91168e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:18.251855 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6c643-69b7-4edf-9bfa-091ba91168e5 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:18.252577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6c643-69b7-4edf-9bfa-091ba91168e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:18.252757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6c643-69b7-4edf-9bfa-091ba91168e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:18.266512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6c643-69b7-4edf-9bfa-091ba91168e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:18.268368 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6c643-69b7-4edf-9bfa-091ba91168e5 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:18.283141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1ae9d-d867-46d9-abd9-89ac28cf6766 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:18.285499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1ae9d-d867-46d9-abd9-89ac28cf6766 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-63a6c643-69b7-4edf-9bfa-091ba91168e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:18.295880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:18.297053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:18.298157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:18.298157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:18.298157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:18.298157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:18.304763 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 335/654] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:18.304763 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6c643-69b7-4edf-9bfa-091ba91168e5 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:18.425596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 320/655] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:38:18.426791 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1ae9d-d867-46d9-abd9-89ac28cf6766 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:18.431738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6c643-69b7-4edf-9bfa-091ba91168e5 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:18.473508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:18.487945 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9e0ee-e702-40b2-a4df-7d0fa03de17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:18.487945 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9e0ee-e702-40b2-a4df-7d0fa03de17d None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:18.487945 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9e0ee-e702-40b2-a4df-7d0fa03de17d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:18.487945 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b9e0ee-e702-40b2-a4df-7d0fa03de17d None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:18.488400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9e0ee-e702-40b2-a4df-7d0fa03de17d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:18.488400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9e0ee-e702-40b2-a4df-7d0fa03de17d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:18.488733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b9e0ee-e702-40b2-a4df-7d0fa03de17d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:18.513035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 336/656] 10.222.0.64 () {56 vars in 1137 bytes} [Thu Jul 14 17:38:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:18.513915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9e0ee-e702-40b2-a4df-7d0fa03de17d None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:18.548146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1ae9d-d867-46d9-abd9-89ac28cf6766 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:18.555862 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:18.593234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7938bc-48cc-4b03-90ec-6e7a7dac0348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:18.594188 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7938bc-48cc-4b03-90ec-6e7a7dac0348 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:18.594574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7938bc-48cc-4b03-90ec-6e7a7dac0348 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:18.594965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7938bc-48cc-4b03-90ec-6e7a7dac0348 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:18.595844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7938bc-48cc-4b03-90ec-6e7a7dac0348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:18.596403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7938bc-48cc-4b03-90ec-6e7a7dac0348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:18.598008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7938bc-48cc-4b03-90ec-6e7a7dac0348 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:18.616552 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 321/657] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:18.616552 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7938bc-48cc-4b03-90ec-6e7a7dac0348 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:18.652778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b9e0ee-e702-40b2-a4df-7d0fa03de17d None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:18.695881 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af2372-ab15-4b44-bfea-b1ea09da9dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:18.696977 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af2372-ab15-4b44-bfea-b1ea09da9dd6 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:18.697573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af2372-ab15-4b44-bfea-b1ea09da9dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:18.697846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af2372-ab15-4b44-bfea-b1ea09da9dd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:18.712947 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af2372-ab15-4b44-bfea-b1ea09da9dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:18.714607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af2372-ab15-4b44-bfea-b1ea09da9dd6 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-b0af2372-ab15-4b44-bfea-b1ea09da9dd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:18.739500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:18.740632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:18.741745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:18.741745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:18.741745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:18.741745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:18.743620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 337/658] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:18.744619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af2372-ab15-4b44-bfea-b1ea09da9dd6 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:18.754533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:18.759780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7938bc-48cc-4b03-90ec-6e7a7dac0348 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:18.798396 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d271d9b5-d7c3-4897-8f46-75485ad4f868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:18.799337 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271d9b5-d7c3-4897-8f46-75485ad4f868 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:18.799863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271d9b5-d7c3-4897-8f46-75485ad4f868 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:18.800377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271d9b5-d7c3-4897-8f46-75485ad4f868 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:18.849199 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:18.868400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af2372-ab15-4b44-bfea-b1ea09da9dd6 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:18.887173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271d9b5-d7c3-4897-8f46-75485ad4f868 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:18.888707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d271d9b5-d7c3-4897-8f46-75485ad4f868 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:18.904731 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681e6dd-be31-4d3b-8f77-c740da16bd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:18.905539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681e6dd-be31-4d3b-8f77-c740da16bd8c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:18.905826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681e6dd-be31-4d3b-8f77-c740da16bd8c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:18.906122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681e6dd-be31-4d3b-8f77-c740da16bd8c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:18.914826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681e6dd-be31-4d3b-8f77-c740da16bd8c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:18.916314 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681e6dd-be31-4d3b-8f77-c740da16bd8c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-2681e6dd-be31-4d3b-8f77-c740da16bd8c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:18.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:18.933559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:18.935046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:18.935046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:18.935046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:18.935046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:18.935046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:18.935046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:18.935046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:18.935046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:18.935046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 17:38:18.935046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:18.935046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 338/659] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:38:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:18.935652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681e6dd-be31-4d3b-8f77-c740da16bd8c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:19.016979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:19.025875 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 322/660] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:38:19.027551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f703050-7203-4f5b-97ff-ba1d31bcd286 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:19.027842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f703050-7203-4f5b-97ff-ba1d31bcd286 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:19.028704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f703050-7203-4f5b-97ff-ba1d31bcd286 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:19.031982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 323/661] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:19.031982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f703050-7203-4f5b-97ff-ba1d31bcd286 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:19.052215 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681e6dd-be31-4d3b-8f77-c740da16bd8c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:19.089608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e7485-ec30-42ec-8e70-fa2df3435e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:19.090553 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e7485-ec30-42ec-8e70-fa2df3435e8b None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:19.091513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e7485-ec30-42ec-8e70-fa2df3435e8b None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:19.091964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e7485-ec30-42ec-8e70-fa2df3435e8b None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:19.093494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e7485-ec30-42ec-8e70-fa2df3435e8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:19.093799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e7485-ec30-42ec-8e70-fa2df3435e8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:19.095207 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e7485-ec30-42ec-8e70-fa2df3435e8b None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:19.108472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 339/662] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:19.109319 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e7485-ec30-42ec-8e70-fa2df3435e8b None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:19.114217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:19.152231 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f703050-7203-4f5b-97ff-ba1d31bcd286 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:19.189367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b5037c-0e58-4641-b74c-0b4d00510867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:19.190046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5037c-0e58-4641-b74c-0b4d00510867 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:19.190325 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5037c-0e58-4641-b74c-0b4d00510867 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:19.190600 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b5037c-0e58-4641-b74c-0b4d00510867 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:19.191436 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5037c-0e58-4641-b74c-0b4d00510867 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:19.191743 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5037c-0e58-4641-b74c-0b4d00510867 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:19.193286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b5037c-0e58-4641-b74c-0b4d00510867 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:19.218305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 324/663] 10.222.0.64 () {56 vars in 1137 bytes} [Thu Jul 14 17:38:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:19.219044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e7485-ec30-42ec-8e70-fa2df3435e8b None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:19.219754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24104bb8-46b6-44ff-8fa1-32580874965f None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:19.220109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24104bb8-46b6-44ff-8fa1-32580874965f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:19.220386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24104bb8-46b6-44ff-8fa1-32580874965f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:19.239821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-24104bb8-46b6-44ff-8fa1-32580874965f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:19.241045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-24104bb8-46b6-44ff-8fa1-32580874965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:19.266401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0efe1-0166-43f7-9b94-61fa062682d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:19.267219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0efe1-0166-43f7-9b94-61fa062682d1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:19.267506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0efe1-0166-43f7-9b94-61fa062682d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:19.267789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae0efe1-0166-43f7-9b94-61fa062682d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:19.280370 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0efe1-0166-43f7-9b94-61fa062682d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:19.282736 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae0efe1-0166-43f7-9b94-61fa062682d1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:19.299834 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-7ae0efe1-0166-43f7-9b94-61fa062682d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:19.302572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:19.303439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:19.304603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:19.304603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:19.304603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:19.304603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:19.304603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 340/664] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:19.305066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0efe1-0166-43f7-9b94-61fa062682d1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:19.320903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 325/665] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:19.322096 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24104bb8-46b6-44ff-8fa1-32580874965f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:19.415109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:19.433152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24104bb8-46b6-44ff-8fa1-32580874965f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:19.468640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae0efe1-0166-43f7-9b94-61fa062682d1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:19.476131 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653dc1b-382d-4115-b1b7-f6660a561180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:19.476968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653dc1b-382d-4115-b1b7-f6660a561180 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:19.477394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653dc1b-382d-4115-b1b7-f6660a561180 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:19.477806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2653dc1b-382d-4115-b1b7-f6660a561180 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:19.512108 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748555c-5852-48da-a737-712dbe7dc713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:19.513297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748555c-5852-48da-a737-712dbe7dc713 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:19.513731 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748555c-5852-48da-a737-712dbe7dc713 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:19.514136 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748555c-5852-48da-a737-712dbe7dc713 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:19.521937 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748555c-5852-48da-a737-712dbe7dc713 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:19.523391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748555c-5852-48da-a737-712dbe7dc713 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-8748555c-5852-48da-a737-712dbe7dc713 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:19.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:19.541241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:19.543302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:19.543302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:19.543302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:19.543302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:19.543302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:19.543302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:19.543302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:19.543302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:19.545683 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 341/666] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:19.546584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748555c-5852-48da-a737-712dbe7dc713 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:19.550021 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:19.566235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653dc1b-382d-4115-b1b7-f6660a561180 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:19.569178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2653dc1b-382d-4115-b1b7-f6660a561180 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:19.657445 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748555c-5852-48da-a737-712dbe7dc713 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:19.676359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:19.701584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1724cf-27e0-4010-9c25-4818991ad9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:19.702240 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1724cf-27e0-4010-9c25-4818991ad9c9 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:19.702531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1724cf-27e0-4010-9c25-4818991ad9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:19.702808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1724cf-27e0-4010-9c25-4818991ad9c9 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:19.703544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1724cf-27e0-4010-9c25-4818991ad9c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:19.703854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1724cf-27e0-4010-9c25-4818991ad9c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:19.705252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1724cf-27e0-4010-9c25-4818991ad9c9 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:19.722803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 342/667] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:19.722803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1724cf-27e0-4010-9c25-4818991ad9c9 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:19.749747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 326/668] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:38:19.749747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653dc1b-382d-4115-b1b7-f6660a561180 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:19.842752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1724cf-27e0-4010-9c25-4818991ad9c9 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:19.879124 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:19.885849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7994bbc-56bc-4efc-beb5-e1d5e07d65ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:19.886660 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7994bbc-56bc-4efc-beb5-e1d5e07d65ab None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:19.887176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7994bbc-56bc-4efc-beb5-e1d5e07d65ab None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:19.887176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7994bbc-56bc-4efc-beb5-e1d5e07d65ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:19.893397 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2653dc1b-382d-4115-b1b7-f6660a561180 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:19.899785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7994bbc-56bc-4efc-beb5-e1d5e07d65ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-f7994bbc-56bc-4efc-beb5-e1d5e07d65ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:19.907655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:19.908658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:19.909541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:19.909541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:19.909541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:19.909541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:19.909541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:19.909541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:19.909541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:19.909541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:19.909541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:19.909541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:19.909541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:38:19.909541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:19.909541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 343/669] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:38:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:19.910244 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7994bbc-56bc-4efc-beb5-e1d5e07d65ab None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:19.937143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86ac65-cc2a-401b-be09-3bc8222d6ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:19.937871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86ac65-cc2a-401b-be09-3bc8222d6ce2 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:19.938104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86ac65-cc2a-401b-be09-3bc8222d6ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:19.938383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a86ac65-cc2a-401b-be09-3bc8222d6ce2 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:19.939175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86ac65-cc2a-401b-be09-3bc8222d6ce2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:19.939491 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86ac65-cc2a-401b-be09-3bc8222d6ce2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:19.940971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a86ac65-cc2a-401b-be09-3bc8222d6ce2 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:19.950955 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:19.958667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 327/670] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:19.958667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86ac65-cc2a-401b-be09-3bc8222d6ce2 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:20.024665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7994bbc-56bc-4efc-beb5-e1d5e07d65ab None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:20.062687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a86ac65-cc2a-401b-be09-3bc8222d6ce2 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:20.064123 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefedb0a-38a4-498b-9bee-63233f25c4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:20.064926 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefedb0a-38a4-498b-9bee-63233f25c4fa None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:20.065212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefedb0a-38a4-498b-9bee-63233f25c4fa None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:20.065486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefedb0a-38a4-498b-9bee-63233f25c4fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:20.072466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefedb0a-38a4-498b-9bee-63233f25c4fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:20.073919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefedb0a-38a4-498b-9bee-63233f25c4fa None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-aefedb0a-38a4-498b-9bee-63233f25c4fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:20.088670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:20.089815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:20.090978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:20.090978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:20.090978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:20.090978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:20.090978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:20.090978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:20.090978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:20.090978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:20.090978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 344/671] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:20.090978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefedb0a-38a4-498b-9bee-63233f25c4fa None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:20.104009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e125e45-6c73-4959-bab8-2190630170d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:20.104744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e125e45-6c73-4959-bab8-2190630170d3 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:20.105031 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e125e45-6c73-4959-bab8-2190630170d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:20.105308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e125e45-6c73-4959-bab8-2190630170d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:20.117310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e125e45-6c73-4959-bab8-2190630170d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:20.118930 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e125e45-6c73-4959-bab8-2190630170d3 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:20.128952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-2e125e45-6c73-4959-bab8-2190630170d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:20.139952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:20.141105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:20.142108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:20.142108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:20.142108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:20.142108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:20.142108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 328/672] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:20.142108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e125e45-6c73-4959-bab8-2190630170d3 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:20.212493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:20.219728 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefedb0a-38a4-498b-9bee-63233f25c4fa None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:20.254175 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93cbf53-7115-4f50-8f2a-ba98f321610d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:20.254852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cbf53-7115-4f50-8f2a-ba98f321610d None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:20.255140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cbf53-7115-4f50-8f2a-ba98f321610d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:20.255420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93cbf53-7115-4f50-8f2a-ba98f321610d None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:20.256211 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cbf53-7115-4f50-8f2a-ba98f321610d None admin] RBAC: Adding query filter params (name=admin) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:20.256524 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cbf53-7115-4f50-8f2a-ba98f321610d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:20.257900 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93cbf53-7115-4f50-8f2a-ba98f321610d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:20.260329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e125e45-6c73-4959-bab8-2190630170d3 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:20.266235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 345/673] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:38:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:20.267526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ea156-b203-40dd-9542-b8895c72459f None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:20.267817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ea156-b203-40dd-9542-b8895c72459f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:20.268139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4ea156-b203-40dd-9542-b8895c72459f None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:20.269403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 346/674] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:20.271578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ea156-b203-40dd-9542-b8895c72459f None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:20.296130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8eec9d-ca75-4e8c-a722-0a0414188453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:20.296790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8eec9d-ca75-4e8c-a722-0a0414188453 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:20.297072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8eec9d-ca75-4e8c-a722-0a0414188453 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:20.297345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8eec9d-ca75-4e8c-a722-0a0414188453 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:20.298175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8eec9d-ca75-4e8c-a722-0a0414188453 None admin] RBAC: Adding query filter params (role.id=2db2bc363aa74930932c823dc291d7f3, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=a5adec9924444e29b8964f7969452592) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:20.298486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8eec9d-ca75-4e8c-a722-0a0414188453 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:20.299956 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8eec9d-ca75-4e8c-a722-0a0414188453 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:20.305734 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 329/675] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:20 2022] GET /identity/v3/role_assignments?user.id=a5adec9924444e29b8964f7969452592&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=2db2bc363aa74930932c823dc291d7f3 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:20.306524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8eec9d-ca75-4e8c-a722-0a0414188453 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:20.388346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:20.397485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4ea156-b203-40dd-9542-b8895c72459f None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:20.433844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8eec9d-ca75-4e8c-a722-0a0414188453 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:20.451275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5c853-1741-4506-bddc-767b342022de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:20.452132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5c853-1741-4506-bddc-767b342022de None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:20.452510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5c853-1741-4506-bddc-767b342022de None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:20.452860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a5c853-1741-4506-bddc-767b342022de None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:20.453779 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5c853-1741-4506-bddc-767b342022de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:20.454173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5c853-1741-4506-bddc-767b342022de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:20.455716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a5c853-1741-4506-bddc-767b342022de None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:20.472742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 347/676] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:20.473770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5c853-1741-4506-bddc-767b342022de None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:20.490167 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479e2ff-309f-40ff-88a0-9c78f01aecfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:20.490974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479e2ff-309f-40ff-88a0-9c78f01aecfc None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:20.491362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479e2ff-309f-40ff-88a0-9c78f01aecfc None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:20.491738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479e2ff-309f-40ff-88a0-9c78f01aecfc None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:20.492602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479e2ff-309f-40ff-88a0-9c78f01aecfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:20.492916 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479e2ff-309f-40ff-88a0-9c78f01aecfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:20.494295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479e2ff-309f-40ff-88a0-9c78f01aecfc None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:20.497583 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:20.507569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 330/677] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:20.508915 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a595a8-0e3f-4b3b-82c2-da000c68225a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:20.509206 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a595a8-0e3f-4b3b-82c2-da000c68225a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:20.509485 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a595a8-0e3f-4b3b-82c2-da000c68225a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:20.527788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-62a595a8-0e3f-4b3b-82c2-da000c68225a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:20.529661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-62a595a8-0e3f-4b3b-82c2-da000c68225a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:20.587586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a5c853-1741-4506-bddc-767b342022de None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:20.612558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 331/678] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:20.612558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a595a8-0e3f-4b3b-82c2-da000c68225a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:20.627826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f7749-e17c-424e-806c-4321a4c869a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:20.628767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f7749-e17c-424e-806c-4321a4c869a1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:20.629121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f7749-e17c-424e-806c-4321a4c869a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:20.629469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22f7749-e17c-424e-806c-4321a4c869a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:20.640810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f7749-e17c-424e-806c-4321a4c869a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:20.642695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22f7749-e17c-424e-806c-4321a4c869a1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-f22f7749-e17c-424e-806c-4321a4c869a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:20.660819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:20.661755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:20.662671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:20.662671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:20.662671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:20.662671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:20.662671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 348/679] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:20.663705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f7749-e17c-424e-806c-4321a4c869a1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:20.681454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:20.735287 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a595a8-0e3f-4b3b-82c2-da000c68225a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:20.741728 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:20.774058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8bea7-d0d1-44b6-8c2b-d6c30877a8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:20.774899 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8bea7-d0d1-44b6-8c2b-d6c30877a8a8 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:20.775254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22f7749-e17c-424e-806c-4321a4c869a1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:20.775362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8bea7-d0d1-44b6-8c2b-d6c30877a8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:20.775726 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8bea7-d0d1-44b6-8c2b-d6c30877a8a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:20.776706 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8bea7-d0d1-44b6-8c2b-d6c30877a8a8 None admin] RBAC: Adding query filter params (role.id=2db2bc363aa74930932c823dc291d7f3, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=b2b4bf2aa1744a46b6d2a1b5a136a0e1) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:20.777084 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8bea7-d0d1-44b6-8c2b-d6c30877a8a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:20.778662 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8bea7-d0d1-44b6-8c2b-d6c30877a8a8 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:20.792303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 332/680] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:20 2022] GET /identity/v3/role_assignments?user.id=b2b4bf2aa1744a46b6d2a1b5a136a0e1&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=2db2bc363aa74930932c823dc291d7f3 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:20.793387 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8bea7-d0d1-44b6-8c2b-d6c30877a8a8 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:20.818332 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955d2f9-53da-43b6-83b9-81544c9c6658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:20.819257 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955d2f9-53da-43b6-83b9-81544c9c6658 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:20.819614 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955d2f9-53da-43b6-83b9-81544c9c6658 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:20.819985 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955d2f9-53da-43b6-83b9-81544c9c6658 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:20.829287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955d2f9-53da-43b6-83b9-81544c9c6658 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:20.830925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955d2f9-53da-43b6-83b9-81544c9c6658 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-f955d2f9-53da-43b6-83b9-81544c9c6658 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:20.849227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:20.850235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:20.851133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:20.851133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:20.851133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:20.851133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:20.851133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:20.851133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:20.851133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:20.851133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:20.851133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 349/681] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:20.852076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955d2f9-53da-43b6-83b9-81544c9c6658 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:20.952423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8bea7-d0d1-44b6-8c2b-d6c30877a8a8 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:20.972252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:20.973695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955d2f9-53da-43b6-83b9-81544c9c6658 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:21.008788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042ddc1-d99a-4fa4-b4b2-f2a468326ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:21.009489 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:21.011383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042ddc1-d99a-4fa4-b4b2-f2a468326ece None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:21.012261 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042ddc1-d99a-4fa4-b4b2-f2a468326ece None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:21.013694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042ddc1-d99a-4fa4-b4b2-f2a468326ece None admin] PATH_INFO: `/v3/roles/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:21.016615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39478900-61f4-4166-94f1-31cf8abd44fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:21.017297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39478900-61f4-4166-94f1-31cf8abd44fc None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:21.017581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39478900-61f4-4166-94f1-31cf8abd44fc None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:21.017858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39478900-61f4-4166-94f1-31cf8abd44fc None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:21.018599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39478900-61f4-4166-94f1-31cf8abd44fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:21.018908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39478900-61f4-4166-94f1-31cf8abd44fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:21.020319 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39478900-61f4-4166-94f1-31cf8abd44fc None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:21.029019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042ddc1-d99a-4fa4-b4b2-f2a468326ece None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:21.039065 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 350/682] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:21.039065 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39478900-61f4-4166-94f1-31cf8abd44fc None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-c042ddc1-d99a-4fa4-b4b2-f2a468326ece None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:21.039336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:21.040211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:21.041065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:21.041065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:21.041065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:21.041065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:21.041065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:21.041065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:21.041065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:21.041065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:21.041065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:21.041065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:21.041065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:21.041065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:21.041065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 333/683] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:21.043818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042ddc1-d99a-4fa4-b4b2-f2a468326ece None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:21.181329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042ddc1-d99a-4fa4-b4b2-f2a468326ece None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:21.186387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39478900-61f4-4166-94f1-31cf8abd44fc None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:21.222561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9852c-4e86-443e-ac5a-984d8ffeeb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:21.223070 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9852c-4e86-443e-ac5a-984d8ffeeb69 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:21.223453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9852c-4e86-443e-ac5a-984d8ffeeb69 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:21.223737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9852c-4e86-443e-ac5a-984d8ffeeb69 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:21.228634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415813f-54fc-4018-a0a6-2d18de965ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:21.230055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415813f-54fc-4018-a0a6-2d18de965ff9 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:21.230544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415813f-54fc-4018-a0a6-2d18de965ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:21.231023 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415813f-54fc-4018-a0a6-2d18de965ff9 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:21.232720 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415813f-54fc-4018-a0a6-2d18de965ff9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:21.233267 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415813f-54fc-4018-a0a6-2d18de965ff9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:21.233601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9852c-4e86-443e-ac5a-984d8ffeeb69 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:21.235193 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9852c-4e86-443e-ac5a-984d8ffeeb69 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:21.235316 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415813f-54fc-4018-a0a6-2d18de965ff9 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-3cd9852c-4e86-443e-ac5a-984d8ffeeb69 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:21.255305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:21.256364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:21.257319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:21.257319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:21.257319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:21.257319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:21.257319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:21.257319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:21.257319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:21.257319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:21.257319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:38:21.257319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:21.257860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 351/684] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:21.257860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415813f-54fc-4018-a0a6-2d18de965ff9 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:21.264580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 334/685] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Jul 14 17:38:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:21.266196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9852c-4e86-443e-ac5a-984d8ffeeb69 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:21.309060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:21.322754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:21.375882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9852c-4e86-443e-ac5a-984d8ffeeb69 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:21.393711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415813f-54fc-4018-a0a6-2d18de965ff9 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:21.424223 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87297da1-ddec-4673-a79e-5d79201feb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:21.425335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87297da1-ddec-4673-a79e-5d79201feb8c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:21.425991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87297da1-ddec-4673-a79e-5d79201feb8c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:21.426462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87297da1-ddec-4673-a79e-5d79201feb8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:21.427589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87297da1-ddec-4673-a79e-5d79201feb8c None admin] RBAC: Adding query filter params (role.id=2db2bc363aa74930932c823dc291d7f3, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=e49fad7db3c643b99e9071d9ca5f0196) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:21.428123 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87297da1-ddec-4673-a79e-5d79201feb8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:21.429765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87297da1-ddec-4673-a79e-5d79201feb8c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:21.431684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63460018-d0f8-4029-8b2f-41187994f309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:21.432720 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63460018-d0f8-4029-8b2f-41187994f309 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:21.433108 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63460018-d0f8-4029-8b2f-41187994f309 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:21.433498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63460018-d0f8-4029-8b2f-41187994f309 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:21.434419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63460018-d0f8-4029-8b2f-41187994f309 None admin] RBAC: Adding query filter params (name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:21.435293 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63460018-d0f8-4029-8b2f-41187994f309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:21.436965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63460018-d0f8-4029-8b2f-41187994f309 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:21.444396 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 335/686] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:21 2022] GET /identity/v3/role_assignments?user.id=e49fad7db3c643b99e9071d9ca5f0196&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=2db2bc363aa74930932c823dc291d7f3 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:21.445664 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87297da1-ddec-4673-a79e-5d79201feb8c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:21.446219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 352/687] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:21.453002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63460018-d0f8-4029-8b2f-41187994f309 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:21.615588 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63460018-d0f8-4029-8b2f-41187994f309 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:21.642438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:21.650591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87297da1-ddec-4673-a79e-5d79201feb8c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:21.652516 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:21.657017 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa766c-339c-4f04-bd4f-a158563c6fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:21.658053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa766c-339c-4f04-bd4f-a158563c6fe4 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:21.658489 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa766c-339c-4f04-bd4f-a158563c6fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:21.659118 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fa766c-339c-4f04-bd4f-a158563c6fe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:21.674348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa766c-339c-4f04-bd4f-a158563c6fe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:21.676793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fa766c-339c-4f04-bd4f-a158563c6fe4 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:21.689905 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8bae4-d976-495a-9b7d-35200cb8a4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:21.690585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8bae4-d976-495a-9b7d-35200cb8a4b7 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:21.690867 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8bae4-d976-495a-9b7d-35200cb8a4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:21.691145 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8bae4-d976-495a-9b7d-35200cb8a4b7 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:21.695581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8bae4-d976-495a-9b7d-35200cb8a4b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:21.695962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8bae4-d976-495a-9b7d-35200cb8a4b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-62fa766c-339c-4f04-bd4f-a158563c6fe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:21.697262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:21.698269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:21.699156 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:21.699156 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:21.699156 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:21.699156 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:21.699394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8bae4-d976-495a-9b7d-35200cb8a4b7 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:21.701416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 353/688] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:21.709631 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa766c-339c-4f04-bd4f-a158563c6fe4 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:21.727210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 336/689] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Jul 14 17:38:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:21.733927 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8bae4-d976-495a-9b7d-35200cb8a4b7 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:21.855545 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fa766c-339c-4f04-bd4f-a158563c6fe4 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:21.860832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8bae4-d976-495a-9b7d-35200cb8a4b7 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:21.898936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0e17f-c4ce-4881-b0ed-ca559b4bed2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:21.899543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0e17f-c4ce-4881-b0ed-ca559b4bed2e None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:21.899809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0e17f-c4ce-4881-b0ed-ca559b4bed2e None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:21.900122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0e17f-c4ce-4881-b0ed-ca559b4bed2e None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:21.900859 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0e17f-c4ce-4881-b0ed-ca559b4bed2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:21.901172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0e17f-c4ce-4881-b0ed-ca559b4bed2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:21.901763 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2493e68-8e3f-443e-90f3-23cf742e16c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:21.902650 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0e17f-c4ce-4881-b0ed-ca559b4bed2e None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:21.902860 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2493e68-8e3f-443e-90f3-23cf742e16c0 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:21.903365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2493e68-8e3f-443e-90f3-23cf742e16c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:21.903830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2493e68-8e3f-443e-90f3-23cf742e16c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:21.916620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 354/690] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:21.925349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0e17f-c4ce-4881-b0ed-ca559b4bed2e None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:21.942069 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:21.958271 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:21.999610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2493e68-8e3f-443e-90f3-23cf742e16c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:22.001816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2493e68-8e3f-443e-90f3-23cf742e16c0 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:22.074568 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0e17f-c4ce-4881-b0ed-ca559b4bed2e None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:22.149755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08858bdc-4e75-4da8-9ddf-74ab6ce1d0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:22.150490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08858bdc-4e75-4da8-9ddf-74ab6ce1d0f6 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:22.150773 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08858bdc-4e75-4da8-9ddf-74ab6ce1d0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:22.151047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08858bdc-4e75-4da8-9ddf-74ab6ce1d0f6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:22.165546 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08858bdc-4e75-4da8-9ddf-74ab6ce1d0f6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:22.167016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08858bdc-4e75-4da8-9ddf-74ab6ce1d0f6 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-08858bdc-4e75-4da8-9ddf-74ab6ce1d0f6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:22.200656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:22.201663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:22.202705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:22.202705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:22.202705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:22.202705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:22.202705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:22.202705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:22.202705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:22.202705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:22.202705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 17:38:22.202705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:22.203245 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 337/691] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:38:22.203318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 355/692] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:38:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:22.207488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2493e68-8e3f-443e-90f3-23cf742e16c0 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:22.209999 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08858bdc-4e75-4da8-9ddf-74ab6ce1d0f6 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:22.317189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:22.330708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:22.349398 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08858bdc-4e75-4da8-9ddf-74ab6ce1d0f6 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:22.388194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a800b8-eb42-42a6-bf0f-5e0378e5a981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:22.388841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a800b8-eb42-42a6-bf0f-5e0378e5a981 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:22.389127 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a800b8-eb42-42a6-bf0f-5e0378e5a981 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:22.389404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a800b8-eb42-42a6-bf0f-5e0378e5a981 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:22.390279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a800b8-eb42-42a6-bf0f-5e0378e5a981 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:22.390669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a800b8-eb42-42a6-bf0f-5e0378e5a981 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:22.390937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2493e68-8e3f-443e-90f3-23cf742e16c0 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:22.392258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a800b8-eb42-42a6-bf0f-5e0378e5a981 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:22.413418 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 356/693] 10.222.0.64 () {56 vars in 1135 bytes} [Thu Jul 14 17:38:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:22.420209 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a800b8-eb42-42a6-bf0f-5e0378e5a981 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:22.433992 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8fb01-b95a-45c9-9e58-4f5f9f6cab36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:22.435427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8fb01-b95a-45c9-9e58-4f5f9f6cab36 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:22.435776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8fb01-b95a-45c9-9e58-4f5f9f6cab36 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:22.436136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8fb01-b95a-45c9-9e58-4f5f9f6cab36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:22.447684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8fb01-b95a-45c9-9e58-4f5f9f6cab36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:22.449300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8fb01-b95a-45c9-9e58-4f5f9f6cab36 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-b1c8fb01-b95a-45c9-9e58-4f5f9f6cab36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:22.474978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:22.475965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:22.476887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:22.476887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:22.476887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:22.476887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:22.476887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 338/694] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:22.482565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8fb01-b95a-45c9-9e58-4f5f9f6cab36 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:22.570705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a800b8-eb42-42a6-bf0f-5e0378e5a981 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:22.607120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aadcd0-6754-4c43-a021-062a4214fdf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:22.607813 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aadcd0-6754-4c43-a021-062a4214fdf0 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:22.608131 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aadcd0-6754-4c43-a021-062a4214fdf0 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:22.608409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aadcd0-6754-4c43-a021-062a4214fdf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:22.621240 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:22.630688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8fb01-b95a-45c9-9e58-4f5f9f6cab36 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:22.652713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:22.678415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c58cb3-a457-475b-ac58-e01cc2e064c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:22.679054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c58cb3-a457-475b-ac58-e01cc2e064c5 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:22.679345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c58cb3-a457-475b-ac58-e01cc2e064c5 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:22.679627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c58cb3-a457-475b-ac58-e01cc2e064c5 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:22.680391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c58cb3-a457-475b-ac58-e01cc2e064c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:22.680742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c58cb3-a457-475b-ac58-e01cc2e064c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:22.682334 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c58cb3-a457-475b-ac58-e01cc2e064c5 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:22.685801 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aadcd0-6754-4c43-a021-062a4214fdf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:22.687513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aadcd0-6754-4c43-a021-062a4214fdf0 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:22.700045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 339/695] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:22.701926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b024d-89b0-445f-91b3-7bc12590d5d0 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:22.703782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b024d-89b0-445f-91b3-7bc12590d5d0 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:22.704913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b024d-89b0-445f-91b3-7bc12590d5d0 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:22.706427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 340/696] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:22.708696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b024d-89b0-445f-91b3-7bc12590d5d0 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:22.827114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 357/697] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 17:38:22.829471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8367d77-9b56-462b-b5ef-fe748ed06fdb None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:22.829968 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8367d77-9b56-462b-b5ef-fe748ed06fdb None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:22.830742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8367d77-9b56-462b-b5ef-fe748ed06fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:22.850927 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-c8367d77-9b56-462b-b5ef-fe748ed06fdb None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:22.852786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c8367d77-9b56-462b-b5ef-fe748ed06fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:22.862598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b024d-89b0-445f-91b3-7bc12590d5d0 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:22.897492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:22.921116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ffd0e-3421-43ca-b0b9-a2b26a76087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:22.922599 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ffd0e-3421-43ca-b0b9-a2b26a76087f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:22.923611 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ffd0e-3421-43ca-b0b9-a2b26a76087f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:22.924128 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ffd0e-3421-43ca-b0b9-a2b26a76087f None admin] PATH_INFO: `/v3/projects/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:22.933187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ffd0e-3421-43ca-b0b9-a2b26a76087f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:22.936309 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274ffd0e-3421-43ca-b0b9-a2b26a76087f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-274ffd0e-3421-43ca-b0b9-a2b26a76087f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:22.955540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:22.956542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:22.957429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:22.957429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:22.957429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:22.957429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:22.957429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:22.957429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:22.957429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:22.957429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:22.964371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 341/699] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:22.964508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 358/699] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:22.965320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8367d77-9b56-462b-b5ef-fe748ed06fdb None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:22.965811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ffd0e-3421-43ca-b0b9-a2b26a76087f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:23.006274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:23.112284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ffd0e-3421-43ca-b0b9-a2b26a76087f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:23.121364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8367d77-9b56-462b-b5ef-fe748ed06fdb None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:23.160989 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d7d2c-817f-495a-b6d3-710a80eaa679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:23.161809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7d2c-817f-495a-b6d3-710a80eaa679 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:23.162093 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7d2c-817f-495a-b6d3-710a80eaa679 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:23.162371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7d2c-817f-495a-b6d3-710a80eaa679 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:23.162853 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fa8c1-6de0-48bd-93ae-47f8b149e66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:23.163766 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fa8c1-6de0-48bd-93ae-47f8b149e66f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:23.164283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fa8c1-6de0-48bd-93ae-47f8b149e66f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:23.165130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fa8c1-6de0-48bd-93ae-47f8b149e66f None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:23.166570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fa8c1-6de0-48bd-93ae-47f8b149e66f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:23.167094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fa8c1-6de0-48bd-93ae-47f8b149e66f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:23.168875 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290fa8c1-6de0-48bd-93ae-47f8b149e66f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:23.174888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d7d2c-817f-495a-b6d3-710a80eaa679 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:23.176424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d7d2c-817f-495a-b6d3-710a80eaa679 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:23.191562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 342/700] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:23.191562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fa8c1-6de0-48bd-93ae-47f8b149e66f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-d40d7d2c-817f-495a-b6d3-710a80eaa679 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:23.203159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:23.204495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:23.205425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:23.205425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:23.205425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:23.205425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:23.205425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 359/701] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:23.205795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d7d2c-817f-495a-b6d3-710a80eaa679 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:23.231919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:23.311407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:23.323201 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290fa8c1-6de0-48bd-93ae-47f8b149e66f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:23.326580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d7d2c-817f-495a-b6d3-710a80eaa679 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:23.370094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919f460-639d-4a7f-878a-f0f36134dae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:23.370344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9495a5-4603-481c-87e8-21c75ef3ecc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:23.370892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919f460-639d-4a7f-878a-f0f36134dae5 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:23.371003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9495a5-4603-481c-87e8-21c75ef3ecc8 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:23.371441 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919f460-639d-4a7f-878a-f0f36134dae5 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:23.371600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9495a5-4603-481c-87e8-21c75ef3ecc8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:23.371678 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919f460-639d-4a7f-878a-f0f36134dae5 None admin] PATH_INFO: `/v3/services/network` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:23.372613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9495a5-4603-481c-87e8-21c75ef3ecc8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:23.372873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919f460-639d-4a7f-878a-f0f36134dae5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:23.373136 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9495a5-4603-481c-87e8-21c75ef3ecc8 None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=6423f54152064b909d03c0908d636993) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:23.373446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9495a5-4603-481c-87e8-21c75ef3ecc8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:23.374250 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b919f460-639d-4a7f-878a-f0f36134dae5 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:23.374872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9495a5-4603-481c-87e8-21c75ef3ecc8 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:23.381156 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 360/702] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:23 2022] GET /identity/v3/role_assignments?user.id=6423f54152064b909d03c0908d636993&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:23.382194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9495a5-4603-481c-87e8-21c75ef3ecc8 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-b919f460-639d-4a7f-878a-f0f36134dae5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:23.392696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:23.393839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:23.394900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:23.394900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:23.394900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:23.394900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:23.394900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:23.394900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:23.394900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:23.394900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:23.394900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:23.394900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:23.394900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:23.394900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:23.394900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 17:38:23.394900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:23.399810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 343/703] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:23 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:23.402524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cbf6b-dcd8-48e1-b5eb-a0a518f4e1a6 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:23.403219 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cbf6b-dcd8-48e1-b5eb-a0a518f4e1a6 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:23.403798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cbf6b-dcd8-48e1-b5eb-a0a518f4e1a6 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:23.405698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 344/704] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:23.406828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cbf6b-dcd8-48e1-b5eb-a0a518f4e1a6 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:23.518964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:23.544260 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9495a5-4603-481c-87e8-21c75ef3ecc8 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:23.545409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cbf6b-dcd8-48e1-b5eb-a0a518f4e1a6 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:23.595982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8000c-b2f6-4135-bbcd-ee66b1432bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:23.595982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8000c-b2f6-4135-bbcd-ee66b1432bb9 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:23.595982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8000c-b2f6-4135-bbcd-ee66b1432bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:23.595982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8000c-b2f6-4135-bbcd-ee66b1432bb9 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:23.596619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8000c-b2f6-4135-bbcd-ee66b1432bb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:23.596619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8000c-b2f6-4135-bbcd-ee66b1432bb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:23.596851 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d13dc-1a9a-4ae6-a863-9b5f71153596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:23.596998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e8000c-b2f6-4135-bbcd-ee66b1432bb9 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:23.597137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d13dc-1a9a-4ae6-a863-9b5f71153596 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:23.597137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d13dc-1a9a-4ae6-a863-9b5f71153596 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:23.597137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d13dc-1a9a-4ae6-a863-9b5f71153596 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:23.597137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d13dc-1a9a-4ae6-a863-9b5f71153596 None admin] RBAC: Adding query filter params (name=network) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:23.597137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d13dc-1a9a-4ae6-a863-9b5f71153596 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:23.597707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d13dc-1a9a-4ae6-a863-9b5f71153596 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:23.605690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 345/705] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Jul 14 17:38:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:23.608700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8378503-0503-4408-85dc-7607cc92bcc8 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:23.608977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8378503-0503-4408-85dc-7607cc92bcc8 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:23.609254 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8378503-0503-4408-85dc-7607cc92bcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:23.611785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 361/706] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:23.619798 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8000c-b2f6-4135-bbcd-ee66b1432bb9 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:23.631592 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b8378503-0503-4408-85dc-7607cc92bcc8 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:23.632872 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b8378503-0503-4408-85dc-7607cc92bcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:23.649663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:23.732837 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 346/707] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:23.734046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8378503-0503-4408-85dc-7607cc92bcc8 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:23.776183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8000c-b2f6-4135-bbcd-ee66b1432bb9 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:23.821149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357efedb-7df2-483c-8b66-8cf1ae166e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:23.821983 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357efedb-7df2-483c-8b66-8cf1ae166e3a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:23.822365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357efedb-7df2-483c-8b66-8cf1ae166e3a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:23.822737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357efedb-7df2-483c-8b66-8cf1ae166e3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:23.830466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357efedb-7df2-483c-8b66-8cf1ae166e3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:23.832111 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357efedb-7df2-483c-8b66-8cf1ae166e3a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-357efedb-7df2-483c-8b66-8cf1ae166e3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:23.848252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:23.849578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:23.851908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:23.851908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:23.851908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:23.851908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:23.851908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:23.851908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:23.851908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:23.851908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:23.851908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 362/708] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:23.853483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357efedb-7df2-483c-8b66-8cf1ae166e3a None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:23.873923 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:23.883543 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8378503-0503-4408-85dc-7607cc92bcc8 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:23.932058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e5ae6-b6ed-44d7-b5a6-f7a231b7cdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:23.933497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e5ae6-b6ed-44d7-b5a6-f7a231b7cdf3 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:23.934237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e5ae6-b6ed-44d7-b5a6-f7a231b7cdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:23.935512 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39e5ae6-b6ed-44d7-b5a6-f7a231b7cdf3 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:23.936399 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e5ae6-b6ed-44d7-b5a6-f7a231b7cdf3 None admin] RBAC: Adding query filter params (type=network) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:23.936760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e5ae6-b6ed-44d7-b5a6-f7a231b7cdf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:23.938555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39e5ae6-b6ed-44d7-b5a6-f7a231b7cdf3 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:23.952349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 347/709] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Jul 14 17:38:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:23.953372 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e5ae6-b6ed-44d7-b5a6-f7a231b7cdf3 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:23.997303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:24.017514 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357efedb-7df2-483c-8b66-8cf1ae166e3a None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:24.097039 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c8889-ddf3-41ae-952c-5856a5c3b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:24.097853 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c8889-ddf3-41ae-952c-5856a5c3b5a6 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:24.098165 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c8889-ddf3-41ae-952c-5856a5c3b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:24.098443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c8889-ddf3-41ae-952c-5856a5c3b5a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:24.121177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c8889-ddf3-41ae-952c-5856a5c3b5a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:24.124314 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c8889-ddf3-41ae-952c-5856a5c3b5a6 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-eb9c8889-ddf3-41ae-952c-5856a5c3b5a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:24.145343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:24.146349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:24.147258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:24.147258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:24.147258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:24.147258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:24.154214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39e5ae6-b6ed-44d7-b5a6-f7a231b7cdf3 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:24.154678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 363/710] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:24.156476 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa84a59-088c-4d06-9c52-402676e717ec None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:24.157016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa84a59-088c-4d06-9c52-402676e717ec None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:24.157494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa84a59-088c-4d06-9c52-402676e717ec None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:24.159695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 364/711] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:24.161206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa84a59-088c-4d06-9c52-402676e717ec None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:24.201473 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68f7b48-c84b-4a27-b485-f9d64c59a323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:24.202163 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f7b48-c84b-4a27-b485-f9d64c59a323 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:24.202443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f7b48-c84b-4a27-b485-f9d64c59a323 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:24.202859 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68f7b48-c84b-4a27-b485-f9d64c59a323 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:24.203523 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f7b48-c84b-4a27-b485-f9d64c59a323 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:24.203834 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f7b48-c84b-4a27-b485-f9d64c59a323 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:24.205646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68f7b48-c84b-4a27-b485-f9d64c59a323 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:24.221436 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 348/712] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:24.234115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e952f5e-15cb-42df-bb86-30523f2e5b6c None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:24.234782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e952f5e-15cb-42df-bb86-30523f2e5b6c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:24.235486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e952f5e-15cb-42df-bb86-30523f2e5b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:24.248329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:24.261209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-2e952f5e-15cb-42df-bb86-30523f2e5b6c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:24.262451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-2e952f5e-15cb-42df-bb86-30523f2e5b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:24.314497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa84a59-088c-4d06-9c52-402676e717ec None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:24.364708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d353059-135a-4c4a-be42-19921848fb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:24.366113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d353059-135a-4c4a-be42-19921848fb93 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:24.366750 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d353059-135a-4c4a-be42-19921848fb93 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:24.367303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d353059-135a-4c4a-be42-19921848fb93 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:24.367797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 349/713] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:24.368709 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e952f5e-15cb-42df-bb86-30523f2e5b6c None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:24.369779 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d353059-135a-4c4a-be42-19921848fb93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:24.370390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d353059-135a-4c4a-be42-19921848fb93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:24.373168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:24.375357 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d353059-135a-4c4a-be42-19921848fb93 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:24.399776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 365/714] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:24.401720 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d353059-135a-4c4a-be42-19921848fb93 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:24.513444 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e952f5e-15cb-42df-bb86-30523f2e5b6c None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:24.553130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d353059-135a-4c4a-be42-19921848fb93 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:24.564187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1560916-666a-4fb5-a7b0-91390771659f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:24.565558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560916-666a-4fb5-a7b0-91390771659f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:24.566213 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560916-666a-4fb5-a7b0-91390771659f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:24.566851 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1560916-666a-4fb5-a7b0-91390771659f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:24.568108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1560916-666a-4fb5-a7b0-91390771659f None admin] RBAC: Adding query filter params (role.id=2db2bc363aa74930932c823dc291d7f3, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=97f15306528e415baa5c69f12f22c75e) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:24.568903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1560916-666a-4fb5-a7b0-91390771659f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:24.571955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1560916-666a-4fb5-a7b0-91390771659f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:24.579048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:24.592532 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 350/715] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:24 2022] GET /identity/v3/role_assignments?user.id=97f15306528e415baa5c69f12f22c75e&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=2db2bc363aa74930932c823dc291d7f3 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:24.594716 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1560916-666a-4fb5-a7b0-91390771659f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:24.596716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc1f27-cab7-4b53-92b9-715ec41d9b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:24.597753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc1f27-cab7-4b53-92b9-715ec41d9b0f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:24.598139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc1f27-cab7-4b53-92b9-715ec41d9b0f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:24.598517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc1f27-cab7-4b53-92b9-715ec41d9b0f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:24.601483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc1f27-cab7-4b53-92b9-715ec41d9b0f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:24.601483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc1f27-cab7-4b53-92b9-715ec41d9b0f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-cfbc1f27-cab7-4b53-92b9-715ec41d9b0f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:24.618627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:24.620117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 17:38:24.621321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:24.624833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 366/716] 10.222.0.64 () {56 vars in 1129 bytes} [Thu Jul 14 17:38:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:24.628274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc1f27-cab7-4b53-92b9-715ec41d9b0f None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:24.773874 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:24.792596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc1f27-cab7-4b53-92b9-715ec41d9b0f None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:24.806049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1560916-666a-4fb5-a7b0-91390771659f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:24.846510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209a11a-0cf6-48cc-98f1-87b217c985f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:24.847190 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209a11a-0cf6-48cc-98f1-87b217c985f2 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:24.847472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209a11a-0cf6-48cc-98f1-87b217c985f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:24.847774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f209a11a-0cf6-48cc-98f1-87b217c985f2 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:24.848787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209a11a-0cf6-48cc-98f1-87b217c985f2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:24.849101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209a11a-0cf6-48cc-98f1-87b217c985f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:24.850695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f209a11a-0cf6-48cc-98f1-87b217c985f2 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:24.856430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 367/717] 10.222.0.64 () {56 vars in 1124 bytes} [Thu Jul 14 17:38:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:24.864201 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209a11a-0cf6-48cc-98f1-87b217c985f2 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:24.885788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcbb757-208a-45a0-ad18-9b1766721a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:24.886407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbb757-208a-45a0-ad18-9b1766721a51 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:24.886694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbb757-208a-45a0-ad18-9b1766721a51 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:24.887008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbb757-208a-45a0-ad18-9b1766721a51 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:24.888944 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbb757-208a-45a0-ad18-9b1766721a51 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:24.890582 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcbb757-208a-45a0-ad18-9b1766721a51 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:24.936062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 351/718] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:38:24 2022] POST /identity/v3/users => generated 254 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:38:24.947176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:25.026082 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f209a11a-0cf6-48cc-98f1-87b217c985f2 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:25.074805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90f199-c88b-41fb-a4ee-0432754383d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:25.075585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90f199-c88b-41fb-a4ee-0432754383d8 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:25.076016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90f199-c88b-41fb-a4ee-0432754383d8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:25.076270 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90f199-c88b-41fb-a4ee-0432754383d8 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:25.077117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90f199-c88b-41fb-a4ee-0432754383d8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:25.077479 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90f199-c88b-41fb-a4ee-0432754383d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:25.079074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b90f199-c88b-41fb-a4ee-0432754383d8 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:25.086607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 368/719] 10.222.0.64 () {56 vars in 1124 bytes} [Thu Jul 14 17:38:24 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:25.888916 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5f90c-3855-45e7-9489-913ff6f22264 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:25.889961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5f90c-3855-45e7-9489-913ff6f22264 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:25.890493 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5f90c-3855-45e7-9489-913ff6f22264 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:25.892412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 352/720] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:25.898359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb9b3a-50fe-49db-b697-a2916177abf1 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:25.899235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb9b3a-50fe-49db-b697-a2916177abf1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:25.899714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb9b3a-50fe-49db-b697-a2916177abf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:25.921052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-1fcb9b3a-50fe-49db-b697-a2916177abf1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:25.922604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1fcb9b3a-50fe-49db-b697-a2916177abf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:26.026744 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 369/721] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:38:26.062012 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5f90c-3855-45e7-9489-913ff6f22264 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:26.212730 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae5f90c-3855-45e7-9489-913ff6f22264 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:26.243553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:26.263956 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8483b99-15e0-46fc-abe1-668735810ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:26.264778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8483b99-15e0-46fc-abe1-668735810ee6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:26.265059 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8483b99-15e0-46fc-abe1-668735810ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:26.265332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8483b99-15e0-46fc-abe1-668735810ee6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:26.280452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8483b99-15e0-46fc-abe1-668735810ee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-a8483b99-15e0-46fc-abe1-668735810ee6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:26.288089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:26.289043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:26.289906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:26.289906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:26.289906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:26.289906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:26.289906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:26.289906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:26.289906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:26.289906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:26.289906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:26.289906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:26.289906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:38:26.289906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:26.293730 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 353/722] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:38:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:26.299323 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb9b3a-50fe-49db-b697-a2916177abf1 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:26.363709 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:26.453035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb9b3a-50fe-49db-b697-a2916177abf1 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:26.498769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a050002-15b1-4059-a289-140a01f2bb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:26.499373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a050002-15b1-4059-a289-140a01f2bb58 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:26.499661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a050002-15b1-4059-a289-140a01f2bb58 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:26.499954 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a050002-15b1-4059-a289-140a01f2bb58 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:26.500725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a050002-15b1-4059-a289-140a01f2bb58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:26.501035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a050002-15b1-4059-a289-140a01f2bb58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:26.502423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a050002-15b1-4059-a289-140a01f2bb58 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:26.516390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 370/723] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:38:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:26.521537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8483b99-15e0-46fc-abe1-668735810ee6 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:26.647470 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8483b99-15e0-46fc-abe1-668735810ee6 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:26.683236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8cfb12-73ce-4af7-b969-ba97969d7f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:26.684081 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cfb12-73ce-4af7-b969-ba97969d7f48 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:26.684368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cfb12-73ce-4af7-b969-ba97969d7f48 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:26.685262 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8cfb12-73ce-4af7-b969-ba97969d7f48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:26.696993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8cfb12-73ce-4af7-b969-ba97969d7f48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:26.698564 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8cfb12-73ce-4af7-b969-ba97969d7f48 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-5a8cfb12-73ce-4af7-b969-ba97969d7f48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:26.717644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:26.718784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:26.719803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:26.719803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:26.719803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:26.719803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:26.719803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 354/724] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:26.725003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8cfb12-73ce-4af7-b969-ba97969d7f48 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:26.725948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29d269-0fe2-4002-9b9a-72b66fd97d51 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:26.726309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29d269-0fe2-4002-9b9a-72b66fd97d51 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:26.726760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29d269-0fe2-4002-9b9a-72b66fd97d51 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:26.732280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 371/725] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:26.735156 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315a611-47bc-44cc-9222-690c07939dc0 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:26.735448 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315a611-47bc-44cc-9222-690c07939dc0 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:26.735732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315a611-47bc-44cc-9222-690c07939dc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:26.773924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-f315a611-47bc-44cc-9222-690c07939dc0 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:26.775067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f315a611-47bc-44cc-9222-690c07939dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:26.791402 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:26.899078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 372/726] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:26.911139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8cfb12-73ce-4af7-b969-ba97969d7f48 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:26.948987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315a611-47bc-44cc-9222-690c07939dc0 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:26.961760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c0f6b-9cad-4f7e-80d8-e2802e3d662e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:26.962522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c0f6b-9cad-4f7e-80d8-e2802e3d662e None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:26.962876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c0f6b-9cad-4f7e-80d8-e2802e3d662e None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:26.963246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c0f6b-9cad-4f7e-80d8-e2802e3d662e None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:26.964075 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c0f6b-9cad-4f7e-80d8-e2802e3d662e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:26.964449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c0f6b-9cad-4f7e-80d8-e2802e3d662e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:26.967859 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c0f6b-9cad-4f7e-80d8-e2802e3d662e None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:26.980610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 355/727] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:26.987190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c0f6b-9cad-4f7e-80d8-e2802e3d662e None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:27.027400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:27.124468 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:27.132095 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c0f6b-9cad-4f7e-80d8-e2802e3d662e None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:27.150754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315a611-47bc-44cc-9222-690c07939dc0 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:27.189712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb3476-e8ca-43e2-b0ff-1c21d380e23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:27.191655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb3476-e8ca-43e2-b0ff-1c21d380e23a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:27.191962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb3476-e8ca-43e2-b0ff-1c21d380e23a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:27.192445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb3476-e8ca-43e2-b0ff-1c21d380e23a None admin] PATH_INFO: `/v3/users/nova` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:27.207236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb3476-e8ca-43e2-b0ff-1c21d380e23a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:27.210594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb3476-e8ca-43e2-b0ff-1c21d380e23a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-4dfb3476-e8ca-43e2-b0ff-1c21d380e23a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:27.230940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:27.232502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:27.233446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:27.233446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:27.233446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:27.233446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:27.233446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:27.233446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:27.233446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:27.233446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:27.233446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:38:27.233446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:27.234843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 356/728] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Jul 14 17:38:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:27.238860 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb3476-e8ca-43e2-b0ff-1c21d380e23a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:27.244652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81425af-731e-4429-86ec-7e20a56bc9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:27.245354 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81425af-731e-4429-86ec-7e20a56bc9f2 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:27.245641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81425af-731e-4429-86ec-7e20a56bc9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:27.245926 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81425af-731e-4429-86ec-7e20a56bc9f2 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:27.246670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81425af-731e-4429-86ec-7e20a56bc9f2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:27.248919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81425af-731e-4429-86ec-7e20a56bc9f2 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:27.280250 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 373/729] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Jul 14 17:38:26 2022] POST /identity/v3/services => generated 240 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:38:27.410654 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb3476-e8ca-43e2-b0ff-1c21d380e23a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:27.483207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66700074-1f8b-48a0-8faa-c5d2b6236101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:27.483948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66700074-1f8b-48a0-8faa-c5d2b6236101 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:27.484253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66700074-1f8b-48a0-8faa-c5d2b6236101 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:27.484534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66700074-1f8b-48a0-8faa-c5d2b6236101 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:27.485427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66700074-1f8b-48a0-8faa-c5d2b6236101 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:27.485742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66700074-1f8b-48a0-8faa-c5d2b6236101 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:27.487270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66700074-1f8b-48a0-8faa-c5d2b6236101 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:27.512989 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 357/730] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Jul 14 17:38:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:27.518817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81425af-731e-4429-86ec-7e20a56bc9f2 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:27.583280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe9f7-1406-41d1-b456-43f16eceaaf8 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:27.583493 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe9f7-1406-41d1-b456-43f16eceaaf8 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:27.583785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe9f7-1406-41d1-b456-43f16eceaaf8 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:27.587965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 358/731] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:27.594806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc69288-3a11-427c-b418-62b672e4b7b4 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:27.595162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc69288-3a11-427c-b418-62b672e4b7b4 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:27.595453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc69288-3a11-427c-b418-62b672e4b7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:27.630154 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-7bc69288-3a11-427c-b418-62b672e4b7b4 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:27.631421 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7bc69288-3a11-427c-b418-62b672e4b7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:27.635413 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:27.685254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:27.701438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81425af-731e-4429-86ec-7e20a56bc9f2 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:27.734180 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 359/732] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:27 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 17:38:27.750768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572f747-7013-4663-9a6b-4a8d5458b6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:27.751701 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572f747-7013-4663-9a6b-4a8d5458b6f4 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:27.752103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572f747-7013-4663-9a6b-4a8d5458b6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:27.752446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572f747-7013-4663-9a6b-4a8d5458b6f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:27.764939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572f747-7013-4663-9a6b-4a8d5458b6f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:27.766509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b572f747-7013-4663-9a6b-4a8d5458b6f4 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:27.782276 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc69288-3a11-427c-b418-62b672e4b7b4 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-b572f747-7013-4663-9a6b-4a8d5458b6f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:27.785516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:27.786516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:27.788550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:27.788550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:27.788550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:27.788550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:27.788550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 374/733] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:27.792381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572f747-7013-4663-9a6b-4a8d5458b6f4 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:27.901455 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b572f747-7013-4663-9a6b-4a8d5458b6f4 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:27.907006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc69288-3a11-427c-b418-62b672e4b7b4 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:27.938212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59deafd1-d75c-4457-b1ea-990f6ff599b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:27.939170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deafd1-d75c-4457-b1ea-990f6ff599b5 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:27.939631 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deafd1-d75c-4457-b1ea-990f6ff599b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:27.940122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deafd1-d75c-4457-b1ea-990f6ff599b5 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:27.941035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59deafd1-d75c-4457-b1ea-990f6ff599b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:27.941506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59deafd1-d75c-4457-b1ea-990f6ff599b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:27.943066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59deafd1-d75c-4457-b1ea-990f6ff599b5 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:27.945997 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db9c55-54df-42ca-87e3-2e1195d34a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:27.947372 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db9c55-54df-42ca-87e3-2e1195d34a00 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:27.947814 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db9c55-54df-42ca-87e3-2e1195d34a00 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:27.948268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56db9c55-54df-42ca-87e3-2e1195d34a00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:27.957061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 375/734] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:27.958061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:27.958899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42bb86-bec9-4565-a1bd-7385eb85ea7b None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:27.959363 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42bb86-bec9-4565-a1bd-7385eb85ea7b None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:27.959789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf42bb86-bec9-4565-a1bd-7385eb85ea7b None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:27.961459 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 376/735] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:27.961724 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56db9c55-54df-42ca-87e3-2e1195d34a00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:27.971488 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351a1e1-73e7-4a28-bf8d-fb6f79be202a None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-56db9c55-54df-42ca-87e3-2e1195d34a00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:27.971688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:27.972807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:27.973754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:27.973754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:27.973754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:27.973754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:27.973754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:27.973754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:27.973754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:27.973754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:27.973754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:27.973754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:27.973754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:38:27.973754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:27.974399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351a1e1-73e7-4a28-bf8d-fb6f79be202a None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:27.974399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351a1e1-73e7-4a28-bf8d-fb6f79be202a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:27.974531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 360/736] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:38:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:27.974531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db9c55-54df-42ca-87e3-2e1195d34a00 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:27.987670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:27.995039 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-a351a1e1-73e7-4a28-bf8d-fb6f79be202a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:27.998141 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a351a1e1-73e7-4a28-bf8d-fb6f79be202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:28.080191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 377/737] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:27 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 17:38:28.081501 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351a1e1-73e7-4a28-bf8d-fb6f79be202a None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:28.086965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56db9c55-54df-42ca-87e3-2e1195d34a00 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:28.124515 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a097f81-7283-4e7b-bc66-fcaba0e4748a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:28.125327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a097f81-7283-4e7b-bc66-fcaba0e4748a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:28.125615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a097f81-7283-4e7b-bc66-fcaba0e4748a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:28.125895 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a097f81-7283-4e7b-bc66-fcaba0e4748a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:28.133581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a097f81-7283-4e7b-bc66-fcaba0e4748a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:28.135073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a097f81-7283-4e7b-bc66-fcaba0e4748a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-4a097f81-7283-4e7b-bc66-fcaba0e4748a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:28.151419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:28.154168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:28.156149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:28.156149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:28.156149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:28.156149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:28.156149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:28.156149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:28.156149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:28.156149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:28.156149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 361/738] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:28.156149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a097f81-7283-4e7b-bc66-fcaba0e4748a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:28.208760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351a1e1-73e7-4a28-bf8d-fb6f79be202a None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:28.220359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:28.246372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907cbc6-cadf-4285-897c-9fc961a09edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:28.247338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907cbc6-cadf-4285-897c-9fc961a09edf None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:28.247625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907cbc6-cadf-4285-897c-9fc961a09edf None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:28.247900 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907cbc6-cadf-4285-897c-9fc961a09edf None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:28.248786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907cbc6-cadf-4285-897c-9fc961a09edf None admin] RBAC: Adding query filter params (name=admin) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:28.249149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907cbc6-cadf-4285-897c-9fc961a09edf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:28.251785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907cbc6-cadf-4285-897c-9fc961a09edf None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:28.255824 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:28.264781 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 378/739] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:38:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:28.265636 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907cbc6-cadf-4285-897c-9fc961a09edf None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:28.270202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a097f81-7283-4e7b-bc66-fcaba0e4748a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:28.308921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09dcbf-86ba-44cd-9e28-fa2d3e062aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:28.309971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09dcbf-86ba-44cd-9e28-fa2d3e062aeb None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:28.310411 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09dcbf-86ba-44cd-9e28-fa2d3e062aeb None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:28.310876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09dcbf-86ba-44cd-9e28-fa2d3e062aeb None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:28.311805 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09dcbf-86ba-44cd-9e28-fa2d3e062aeb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:28.313619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09dcbf-86ba-44cd-9e28-fa2d3e062aeb None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:28.330596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 362/740] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Jul 14 17:38:28 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:38:28.331669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09dcbf-86ba-44cd-9e28-fa2d3e062aeb None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:28.391901 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907cbc6-cadf-4285-897c-9fc961a09edf None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:28.452452 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2074882c-2173-4245-ad2c-a11d7b6a4ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:28.453575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074882c-2173-4245-ad2c-a11d7b6a4ac7 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:28.454070 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074882c-2173-4245-ad2c-a11d7b6a4ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:28.454579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2074882c-2173-4245-ad2c-a11d7b6a4ac7 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:28.455789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074882c-2173-4245-ad2c-a11d7b6a4ac7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:28.456403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074882c-2173-4245-ad2c-a11d7b6a4ac7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:28.458173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2074882c-2173-4245-ad2c-a11d7b6a4ac7 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:28.478022 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 379/741] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:28.479827 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d459f8-bb3b-4a3d-a98d-5b9b3067612d None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:28.480451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d459f8-bb3b-4a3d-a98d-5b9b3067612d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:28.480958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d459f8-bb3b-4a3d-a98d-5b9b3067612d None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:28.483489 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 380/742] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:28.484583 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d459f8-bb3b-4a3d-a98d-5b9b3067612d None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:28.492684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09dcbf-86ba-44cd-9e28-fa2d3e062aeb None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:28.516675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:28.541074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70df1ae2-6e36-4561-b267-4e913c879594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:28.542332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df1ae2-6e36-4561-b267-4e913c879594 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:28.542623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df1ae2-6e36-4561-b267-4e913c879594 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:28.542908 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70df1ae2-6e36-4561-b267-4e913c879594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:28.555834 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df1ae2-6e36-4561-b267-4e913c879594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:28.557144 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70df1ae2-6e36-4561-b267-4e913c879594 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:28.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-70df1ae2-6e36-4561-b267-4e913c879594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:28.581178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:28.582177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:28.583065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:28.583065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:28.583065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:28.583065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:28.585326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 363/743] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:28.586787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d935138-8cc1-485e-9b71-03866f7937b5 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:28.587264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d935138-8cc1-485e-9b71-03866f7937b5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:28.587755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d935138-8cc1-485e-9b71-03866f7937b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:28.605691 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-5d935138-8cc1-485e-9b71-03866f7937b5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:28.607345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5d935138-8cc1-485e-9b71-03866f7937b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:28.616589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d459f8-bb3b-4a3d-a98d-5b9b3067612d None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:28.657144 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5613605-2a2c-4469-be60-5675e04ac2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:28.658193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5613605-2a2c-4469-be60-5675e04ac2a1 None admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:28.658494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5613605-2a2c-4469-be60-5675e04ac2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:28.658771 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5613605-2a2c-4469-be60-5675e04ac2a1 None admin] PATH_INFO: `/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/6423f54152064b909d03c0908d636993/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:28.661133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5613605-2a2c-4469-be60-5675e04ac2a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=de62d2fb17f846f2bce5bef108353fd8, user_id=6423f54152064b909d03c0908d636993, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:28.662662 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5613605-2a2c-4469-be60-5675e04ac2a1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:28.680727 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 381/744] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:38:28 2022] PUT /identity/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/6423f54152064b909d03c0908d636993/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:38:28.684167 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5613605-2a2c-4469-be60-5675e04ac2a1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:28.691968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 364/745] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:28.748958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d935138-8cc1-485e-9b71-03866f7937b5 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:28.883995 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5613605-2a2c-4469-be60-5675e04ac2a1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:28.899770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:28.927266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72fdff-86e7-4468-b0fc-4a5534710c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:28.928058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72fdff-86e7-4468-b0fc-4a5534710c2a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:28.928268 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72fdff-86e7-4468-b0fc-4a5534710c2a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:28.928546 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72fdff-86e7-4468-b0fc-4a5534710c2a None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:28.929856 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72fdff-86e7-4468-b0fc-4a5534710c2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:28.930171 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72fdff-86e7-4468-b0fc-4a5534710c2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:28.931969 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72fdff-86e7-4468-b0fc-4a5534710c2a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:28.952016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 382/746] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:28.959756 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72fdff-86e7-4468-b0fc-4a5534710c2a None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:28.981810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d935138-8cc1-485e-9b71-03866f7937b5 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:28.983711 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:29.038819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf93178-5591-4bcd-8764-e138b6687526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:29.039600 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf93178-5591-4bcd-8764-e138b6687526 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:29.039891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf93178-5591-4bcd-8764-e138b6687526 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:29.040196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf93178-5591-4bcd-8764-e138b6687526 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:29.058292 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf93178-5591-4bcd-8764-e138b6687526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-8cf93178-5591-4bcd-8764-e138b6687526 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:29.066055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:29.067006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:29.067973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:29.067973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:29.067973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:29.067973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:29.067973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:29.067973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:29.067973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:29.067973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:29.067973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:29.067973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:29.067973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:29.067973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:29.083445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 365/747] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:29.084439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf93178-5591-4bcd-8764-e138b6687526 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:29.161688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72fdff-86e7-4468-b0fc-4a5534710c2a None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:29.236677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0949e2d-aa01-4928-9901-d5120c68d648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:29.237502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0949e2d-aa01-4928-9901-d5120c68d648 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:29.237784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0949e2d-aa01-4928-9901-d5120c68d648 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:29.238058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0949e2d-aa01-4928-9901-d5120c68d648 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:29.245974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0949e2d-aa01-4928-9901-d5120c68d648 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:29.246643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf93178-5591-4bcd-8764-e138b6687526 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:29.247704 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0949e2d-aa01-4928-9901-d5120c68d648 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-c0949e2d-aa01-4928-9901-d5120c68d648 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:29.262766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:29.264942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:29.266808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:29.266808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:29.266808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:29.266808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:29.266808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:29.266808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:29.266808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:29.266808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:29.266808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 17:38:29.266808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:29.266808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 383/748] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:38:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:29.269896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0949e2d-aa01-4928-9901-d5120c68d648 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:29.292003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50f5e0-2468-4bda-bebf-a18237da592b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:29.293314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50f5e0-2468-4bda-bebf-a18237da592b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:29.293817 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50f5e0-2468-4bda-bebf-a18237da592b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:29.294855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50f5e0-2468-4bda-bebf-a18237da592b None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:29.296451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50f5e0-2468-4bda-bebf-a18237da592b None admin] RBAC: Adding query filter params (name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:29.297272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50f5e0-2468-4bda-bebf-a18237da592b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:29.299778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50f5e0-2468-4bda-bebf-a18237da592b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:29.307567 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:29.312266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 366/749] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:29.317496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50f5e0-2468-4bda-bebf-a18237da592b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:29.347696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:29.436772 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50f5e0-2468-4bda-bebf-a18237da592b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:29.459138 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0949e2d-aa01-4928-9901-d5120c68d648 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:29.477344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a952c-08d7-4fb8-bb69-5ceac99be66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:29.479029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a952c-08d7-4fb8-bb69-5ceac99be66e None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:29.479647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a952c-08d7-4fb8-bb69-5ceac99be66e None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:29.480435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149a952c-08d7-4fb8-bb69-5ceac99be66e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:29.495277 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a952c-08d7-4fb8-bb69-5ceac99be66e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:29.497661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149a952c-08d7-4fb8-bb69-5ceac99be66e None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-149a952c-08d7-4fb8-bb69-5ceac99be66e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:29.519951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:29.520914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:29.521811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:29.521811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:29.521811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:29.521811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:29.524353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 367/750] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:29.530578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a952c-08d7-4fb8-bb69-5ceac99be66e None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:29.539126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080027b5-9664-4da7-a16d-b220ee445dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:29.540259 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080027b5-9664-4da7-a16d-b220ee445dab None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:29.547509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080027b5-9664-4da7-a16d-b220ee445dab None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:29.548150 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080027b5-9664-4da7-a16d-b220ee445dab None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:29.549339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080027b5-9664-4da7-a16d-b220ee445dab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:29.549892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080027b5-9664-4da7-a16d-b220ee445dab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:29.551700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080027b5-9664-4da7-a16d-b220ee445dab None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:29.606334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 384/751] 10.222.0.64 () {56 vars in 1135 bytes} [Thu Jul 14 17:38:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:29.617168 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080027b5-9664-4da7-a16d-b220ee445dab None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:29.660307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:29.682601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149a952c-08d7-4fb8-bb69-5ceac99be66e None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:29.723499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e2f3f-461e-49ea-94d8-f7df1ad2ed21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:29.724825 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e2f3f-461e-49ea-94d8-f7df1ad2ed21 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:29.725458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e2f3f-461e-49ea-94d8-f7df1ad2ed21 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:29.726181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e2f3f-461e-49ea-94d8-f7df1ad2ed21 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:29.727451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e2f3f-461e-49ea-94d8-f7df1ad2ed21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:29.728173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e2f3f-461e-49ea-94d8-f7df1ad2ed21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:29.730003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e2f3f-461e-49ea-94d8-f7df1ad2ed21 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:29.745682 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 368/752] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:29.756034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e2f3f-461e-49ea-94d8-f7df1ad2ed21 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:29.874475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:29.889996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080027b5-9664-4da7-a16d-b220ee445dab None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:29.953799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fba94b-358c-4f03-b677-a2b83353b21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:29.954517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fba94b-358c-4f03-b677-a2b83353b21f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:29.954785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fba94b-358c-4f03-b677-a2b83353b21f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:29.954991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fba94b-358c-4f03-b677-a2b83353b21f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:29.976876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e2f3f-461e-49ea-94d8-f7df1ad2ed21 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:30.051326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237bfd0-3258-47ac-8981-7b1f8d93a387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:30.052655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237bfd0-3258-47ac-8981-7b1f8d93a387 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:30.053104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237bfd0-3258-47ac-8981-7b1f8d93a387 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:30.053530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237bfd0-3258-47ac-8981-7b1f8d93a387 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:30.059104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:30.068904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237bfd0-3258-47ac-8981-7b1f8d93a387 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:30.071437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237bfd0-3258-47ac-8981-7b1f8d93a387 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:30.079021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fba94b-358c-4f03-b677-a2b83353b21f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:30.082502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fba94b-358c-4f03-b677-a2b83353b21f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-f237bfd0-3258-47ac-8981-7b1f8d93a387 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:30.098636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:30.100505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:30.102330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:30.102330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:30.102330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:30.102330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:30.102330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:30.102330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:30.102330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:30.102330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:30.102330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 17:38:30.102330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:30.102330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 369/753] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Jul 14 17:38:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:30.103479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00983aca-9d90-4ef6-ad7c-ff131d28d39e None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:30.111478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00983aca-9d90-4ef6-ad7c-ff131d28d39e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:30.112135 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00983aca-9d90-4ef6-ad7c-ff131d28d39e None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:30.114885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 370/754] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:30 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:30.116257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00983aca-9d90-4ef6-ad7c-ff131d28d39e None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:30.224537 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:30.232333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 385/755] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:38:30.233693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf4fcf-7597-464e-9e3d-3d6da4be8d9e None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:30.234029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf4fcf-7597-464e-9e3d-3d6da4be8d9e None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:30.234321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cf4fcf-7597-464e-9e3d-3d6da4be8d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:30.240384 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00983aca-9d90-4ef6-ad7c-ff131d28d39e None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:30.253203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-a6cf4fcf-7597-464e-9e3d-3d6da4be8d9e None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:30.254558 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a6cf4fcf-7597-464e-9e3d-3d6da4be8d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:30.292483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af4625-a026-4b46-94f0-780a00571b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:30.293336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af4625-a026-4b46-94f0-780a00571b5b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:30.294484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af4625-a026-4b46-94f0-780a00571b5b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:30.294788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af4625-a026-4b46-94f0-780a00571b5b None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:30.295659 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af4625-a026-4b46-94f0-780a00571b5b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:30.296021 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af4625-a026-4b46-94f0-780a00571b5b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:30.297548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af4625-a026-4b46-94f0-780a00571b5b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:30.327484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 371/756] 10.222.0.64 () {56 vars in 1149 bytes} [Thu Jul 14 17:38:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:30.328478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af4625-a026-4b46-94f0-780a00571b5b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:30.344564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 386/757] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:30.346103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf4fcf-7597-464e-9e3d-3d6da4be8d9e None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:30.391058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:30.450228 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af4625-a026-4b46-94f0-780a00571b5b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:30.484534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cf4fcf-7597-464e-9e3d-3d6da4be8d9e None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:30.507283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999f3d-35ca-44a8-9f4c-cf8b40f5c3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:30.508146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999f3d-35ca-44a8-9f4c-cf8b40f5c3b4 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:30.509189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999f3d-35ca-44a8-9f4c-cf8b40f5c3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:30.509525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e999f3d-35ca-44a8-9f4c-cf8b40f5c3b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:30.512165 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:30.528202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7ae2ef-7a26-4991-89aa-be470a7b74aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:30.529159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ae2ef-7a26-4991-89aa-be470a7b74aa None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:30.529512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ae2ef-7a26-4991-89aa-be470a7b74aa None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:30.529885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7ae2ef-7a26-4991-89aa-be470a7b74aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:30.531114 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e999f3d-35ca-44a8-9f4c-cf8b40f5c3b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:30.532824 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e999f3d-35ca-44a8-9f4c-cf8b40f5c3b4 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-4e999f3d-35ca-44a8-9f4c-cf8b40f5c3b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:30.559047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:30.560692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:30.561607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:30.561607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:30.561607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:30.561607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:30.564016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 372/758] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:30.566009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999f3d-35ca-44a8-9f4c-cf8b40f5c3b4 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:30.634360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ae2ef-7a26-4991-89aa-be470a7b74aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:30.636575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7ae2ef-7a26-4991-89aa-be470a7b74aa None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:30.679724 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e999f3d-35ca-44a8-9f4c-cf8b40f5c3b4 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:30.688368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:30.719088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ddcec2-1459-4aba-8712-db6c0183e8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:30.719801 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddcec2-1459-4aba-8712-db6c0183e8f6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:30.720119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddcec2-1459-4aba-8712-db6c0183e8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:30.720549 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ddcec2-1459-4aba-8712-db6c0183e8f6 None admin] PATH_INFO: `/v3/services/network` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:30.721076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ddcec2-1459-4aba-8712-db6c0183e8f6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:30.722466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ddcec2-1459-4aba-8712-db6c0183e8f6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-64ddcec2-1459-4aba-8712-db6c0183e8f6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:30.737755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:30.738926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:30.739778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:30.739778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:30.739778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:30.739778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:30.739778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:30.739778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:30.739778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:30.739778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:30.739778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:30.739778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:30.739778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:30.739778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:30.739778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 17:38:30.739778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:30.745070 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 373/759] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:30 2022] GET /identity/v3/services/network => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:30.745859 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ddcec2-1459-4aba-8712-db6c0183e8f6 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:30.768042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 387/760] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:38:30.768042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f5229-6293-4171-8d4d-b7b4c206363e None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:30.768042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f5229-6293-4171-8d4d-b7b4c206363e None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:30.768042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f5229-6293-4171-8d4d-b7b4c206363e None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:30.768042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 388/761] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:30.768760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f5229-6293-4171-8d4d-b7b4c206363e None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:30.809721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:30.884529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ddcec2-1459-4aba-8712-db6c0183e8f6 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:30.913984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15f5229-6293-4171-8d4d-b7b4c206363e None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:30.922503 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21e12f-3edf-43ff-bd6c-30e5672755f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:30.923188 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21e12f-3edf-43ff-bd6c-30e5672755f5 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:30.923475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21e12f-3edf-43ff-bd6c-30e5672755f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:30.923757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21e12f-3edf-43ff-bd6c-30e5672755f5 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:30.924535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21e12f-3edf-43ff-bd6c-30e5672755f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:30.924852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21e12f-3edf-43ff-bd6c-30e5672755f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:30.926238 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a21e12f-3edf-43ff-bd6c-30e5672755f5 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:30.942392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 374/762] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:30.943054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21e12f-3edf-43ff-bd6c-30e5672755f5 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:30.961256 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03380889-0300-45a7-b278-a04dcb8907d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:30.961934 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03380889-0300-45a7-b278-a04dcb8907d3 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:30.962240 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03380889-0300-45a7-b278-a04dcb8907d3 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:30.962489 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03380889-0300-45a7-b278-a04dcb8907d3 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:30.963213 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03380889-0300-45a7-b278-a04dcb8907d3 None admin] RBAC: Adding query filter params (name=network) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:30.963523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03380889-0300-45a7-b278-a04dcb8907d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:30.964950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03380889-0300-45a7-b278-a04dcb8907d3 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:30.970032 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 389/763] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Jul 14 17:38:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:30.971295 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622be50-1b38-4644-a2e9-d5e37e8f8362 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:30.971544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622be50-1b38-4644-a2e9-d5e37e8f8362 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:30.971799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622be50-1b38-4644-a2e9-d5e37e8f8362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:30.989100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-d622be50-1b38-4644-a2e9-d5e37e8f8362 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:30.989271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:30.990223 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d622be50-1b38-4644-a2e9-d5e37e8f8362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:31.068110 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 390/764] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:31.069541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622be50-1b38-4644-a2e9-d5e37e8f8362 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:31.085172 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a21e12f-3edf-43ff-bd6c-30e5672755f5 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:31.099041 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:31.133466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cf79e7-1059-4d48-9683-86daa63a977a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:31.134287 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf79e7-1059-4d48-9683-86daa63a977a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:31.134909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf79e7-1059-4d48-9683-86daa63a977a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:31.135198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cf79e7-1059-4d48-9683-86daa63a977a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:31.147155 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cf79e7-1059-4d48-9683-86daa63a977a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:31.148785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cf79e7-1059-4d48-9683-86daa63a977a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-f2cf79e7-1059-4d48-9683-86daa63a977a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:31.166847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:31.167784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:31.174086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:31.174086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:31.174086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:31.174086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:31.174086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 375/765] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:31.174086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cf79e7-1059-4d48-9683-86daa63a977a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:31.202136 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d622be50-1b38-4644-a2e9-d5e37e8f8362 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:31.240434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78896a-2663-4235-9ed3-1bce414107a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:31.241151 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78896a-2663-4235-9ed3-1bce414107a5 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:31.241439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78896a-2663-4235-9ed3-1bce414107a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:31.241719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a78896a-2663-4235-9ed3-1bce414107a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:31.249399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78896a-2663-4235-9ed3-1bce414107a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:31.250872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a78896a-2663-4235-9ed3-1bce414107a5 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:31.260690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-2a78896a-2663-4235-9ed3-1bce414107a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:31.265094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:31.266047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:31.266961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:31.266961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:31.266961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:31.266961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:31.266961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:31.266961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:31.266961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:31.266961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:31.266961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 391/766] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:31.267436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78896a-2663-4235-9ed3-1bce414107a5 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:31.280229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cf79e7-1059-4d48-9683-86daa63a977a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:31.315346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a234395-0113-4930-8d36-5219b5d3bd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:31.316042 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a234395-0113-4930-8d36-5219b5d3bd3b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:31.316325 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a234395-0113-4930-8d36-5219b5d3bd3b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:31.316599 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a234395-0113-4930-8d36-5219b5d3bd3b None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:31.317302 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a234395-0113-4930-8d36-5219b5d3bd3b None admin] RBAC: Adding query filter params (type=network) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:31.317605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a234395-0113-4930-8d36-5219b5d3bd3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:31.318969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a234395-0113-4930-8d36-5219b5d3bd3b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:31.323810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 376/767] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Jul 14 17:38:31 2022] GET /identity/v3/services?type=network => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:31.324623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a234395-0113-4930-8d36-5219b5d3bd3b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:31.350737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:31.375774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a78896a-2663-4235-9ed3-1bce414107a5 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:31.409747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb4594e-0525-45fa-951c-76ac626244f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:31.410558 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4594e-0525-45fa-951c-76ac626244f4 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:31.410847 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4594e-0525-45fa-951c-76ac626244f4 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:31.411122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb4594e-0525-45fa-951c-76ac626244f4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:31.411781 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4594e-0525-45fa-951c-76ac626244f4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:31.413200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb4594e-0525-45fa-951c-76ac626244f4 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:31.423011 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a234395-0113-4930-8d36-5219b5d3bd3b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-3cb4594e-0525-45fa-951c-76ac626244f4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:31.425894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:31.426956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:31.427850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 392/768] 10.222.0.64 () {56 vars in 1129 bytes} [Thu Jul 14 17:38:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:31.428979 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5324054-8e0e-424a-928e-e60dd907d740 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:31.428979 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5324054-8e0e-424a-928e-e60dd907d740 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:31.428979 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5324054-8e0e-424a-928e-e60dd907d740 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:31.431981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 393/769] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:31.431981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5324054-8e0e-424a-928e-e60dd907d740 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:31.461215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decab95a-4fa0-4c7a-994a-889f09e6b242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:31.467961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decab95a-4fa0-4c7a-994a-889f09e6b242 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:31.467961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decab95a-4fa0-4c7a-994a-889f09e6b242 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:31.467961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decab95a-4fa0-4c7a-994a-889f09e6b242 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:31.467961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decab95a-4fa0-4c7a-994a-889f09e6b242 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:31.467961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decab95a-4fa0-4c7a-994a-889f09e6b242 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:31.467961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decab95a-4fa0-4c7a-994a-889f09e6b242 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:31.478141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 377/770] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:31.478983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decab95a-4fa0-4c7a-994a-889f09e6b242 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:31.488435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:31.540652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5324054-8e0e-424a-928e-e60dd907d740 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:31.574662 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6346e10-57dc-4b17-83d8-4ce64faf883a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:31.575353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6346e10-57dc-4b17-83d8-4ce64faf883a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:31.575636 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6346e10-57dc-4b17-83d8-4ce64faf883a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:31.575910 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6346e10-57dc-4b17-83d8-4ce64faf883a None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:31.576795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6346e10-57dc-4b17-83d8-4ce64faf883a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:31.577108 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6346e10-57dc-4b17-83d8-4ce64faf883a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:31.578541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6346e10-57dc-4b17-83d8-4ce64faf883a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:31.580333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decab95a-4fa0-4c7a-994a-889f09e6b242 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:31.587947 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:31.592725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 394/771] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:31.593721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6346e10-57dc-4b17-83d8-4ce64faf883a None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:31.618948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a95e675-c75d-436a-bc9c-005aa4d71e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:31.620127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95e675-c75d-436a-bc9c-005aa4d71e5f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:31.620660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95e675-c75d-436a-bc9c-005aa4d71e5f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:31.621152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a95e675-c75d-436a-bc9c-005aa4d71e5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:31.622778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95e675-c75d-436a-bc9c-005aa4d71e5f None admin] RBAC: Adding query filter params (interface=public, service_id=5d4f323f1fab4324a8445676ce9033d4, region_id=RegionOne) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:31.623292 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95e675-c75d-436a-bc9c-005aa4d71e5f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:31.625761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a95e675-c75d-436a-bc9c-005aa4d71e5f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:31.633675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 378/772] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Jul 14 17:38:31 2022] GET /identity/v3/endpoints?service_id=5d4f323f1fab4324a8445676ce9033d4&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:31.634897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd569702-f97d-49a6-82a7-11517e9527d2 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:31.635185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd569702-f97d-49a6-82a7-11517e9527d2 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:31.635467 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd569702-f97d-49a6-82a7-11517e9527d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:31.651858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-fd569702-f97d-49a6-82a7-11517e9527d2 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:31.653131 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-fd569702-f97d-49a6-82a7-11517e9527d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:31.695669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6346e10-57dc-4b17-83d8-4ce64faf883a None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:31.717580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:31.723998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 379/773] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:31.724838 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd569702-f97d-49a6-82a7-11517e9527d2 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:31.734992 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e927048-057a-46db-bfab-b0cef6910bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:31.735889 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e927048-057a-46db-bfab-b0cef6910bdb None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:31.736305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e927048-057a-46db-bfab-b0cef6910bdb None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:31.736843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e927048-057a-46db-bfab-b0cef6910bdb None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:31.737676 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e927048-057a-46db-bfab-b0cef6910bdb None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:31.738094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e927048-057a-46db-bfab-b0cef6910bdb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:31.739677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e927048-057a-46db-bfab-b0cef6910bdb None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:31.750975 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 395/774] 10.222.0.64 () {56 vars in 1124 bytes} [Thu Jul 14 17:38:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:31.750975 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e927048-057a-46db-bfab-b0cef6910bdb None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:31.833984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:31.853726 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd569702-f97d-49a6-82a7-11517e9527d2 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:31.877344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e927048-057a-46db-bfab-b0cef6910bdb None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:31.893676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec77f1-4ba3-46e4-b3c2-37ac7b8d4d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:31.894492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec77f1-4ba3-46e4-b3c2-37ac7b8d4d90 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:31.894781 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec77f1-4ba3-46e4-b3c2-37ac7b8d4d90 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:31.895057 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ec77f1-4ba3-46e4-b3c2-37ac7b8d4d90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:31.902133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec77f1-4ba3-46e4-b3c2-37ac7b8d4d90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:31.903655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ec77f1-4ba3-46e4-b3c2-37ac7b8d4d90 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:31.913690 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b977c7-7449-49c7-84fb-b3889dbd8a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:31.914435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b977c7-7449-49c7-84fb-b3889dbd8a74 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:31.914720 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b977c7-7449-49c7-84fb-b3889dbd8a74 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:31.914998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b977c7-7449-49c7-84fb-b3889dbd8a74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:31.915837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b977c7-7449-49c7-84fb-b3889dbd8a74 None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=97f15306528e415baa5c69f12f22c75e) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:31.916181 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b977c7-7449-49c7-84fb-b3889dbd8a74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:31.917619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b977c7-7449-49c7-84fb-b3889dbd8a74 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-60ec77f1-4ba3-46e4-b3c2-37ac7b8d4d90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:31.917712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:31.918980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:31.920752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:31.920752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:31.920752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:31.920752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:31.920752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:31.920752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:31.920752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:31.920752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:31.920752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 380/775] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:31.920752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec77f1-4ba3-46e4-b3c2-37ac7b8d4d90 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:31.939823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 396/776] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:31 2022] GET /identity/v3/role_assignments?user.id=97f15306528e415baa5c69f12f22c75e&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:31.939823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b977c7-7449-49c7-84fb-b3889dbd8a74 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:32.006273 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:32.063320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ec77f1-4ba3-46e4-b3c2-37ac7b8d4d90 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:32.064559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b977c7-7449-49c7-84fb-b3889dbd8a74 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:32.100834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:32.106227 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37747c-a461-4e99-9de3-f6c739e86161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:32.107555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37747c-a461-4e99-9de3-f6c739e86161 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:32.107904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37747c-a461-4e99-9de3-f6c739e86161 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:32.108273 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37747c-a461-4e99-9de3-f6c739e86161 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:32.108911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37747c-a461-4e99-9de3-f6c739e86161 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:32.110308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37747c-a461-4e99-9de3-f6c739e86161 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:32.112423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025468b-921c-4b51-98d4-ecfa89b7ce04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:32.113106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025468b-921c-4b51-98d4-ecfa89b7ce04 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:32.113391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025468b-921c-4b51-98d4-ecfa89b7ce04 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:32.114389 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025468b-921c-4b51-98d4-ecfa89b7ce04 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:32.115122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025468b-921c-4b51-98d4-ecfa89b7ce04 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:32.115903 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025468b-921c-4b51-98d4-ecfa89b7ce04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:32.117355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025468b-921c-4b51-98d4-ecfa89b7ce04 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:32.127298 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 397/777] 10.222.0.64 () {56 vars in 1124 bytes} [Thu Jul 14 17:38:31 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:32.127886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 381/778] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Jul 14 17:38:31 2022] GET /identity/v3/services => generated 772 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:32.128258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025468b-921c-4b51-98d4-ecfa89b7ce04 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:32.129155 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37747c-a461-4e99-9de3-f6c739e86161 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:32.261753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025468b-921c-4b51-98d4-ecfa89b7ce04 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:32.301734 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37747c-a461-4e99-9de3-f6c739e86161 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:32.324077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9c724-266a-45d6-b392-5ef7f96db901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:32.325784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9c724-266a-45d6-b392-5ef7f96db901 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:32.326252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9c724-266a-45d6-b392-5ef7f96db901 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:32.326764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9c724-266a-45d6-b392-5ef7f96db901 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:32.333888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:32.349373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9c724-266a-45d6-b392-5ef7f96db901 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:32.357981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d619d2-54eb-46af-a96d-395d749f9ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-e7b9c724-266a-45d6-b392-5ef7f96db901 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:32.358987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:32.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:32.361177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:32.361177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:32.361177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:32.361177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:32.361177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:32.361177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:32.361177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:32.361177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:32.361177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:32.361177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:32.361177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:38:32.361177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:32.361177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 398/779] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:38:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:32.361858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9c724-266a-45d6-b392-5ef7f96db901 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:32.362209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d619d2-54eb-46af-a96d-395d749f9ae5 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:32.362857 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d619d2-54eb-46af-a96d-395d749f9ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:32.363344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d619d2-54eb-46af-a96d-395d749f9ae5 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:32.364380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d619d2-54eb-46af-a96d-395d749f9ae5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:32.364901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d619d2-54eb-46af-a96d-395d749f9ae5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:32.366792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d619d2-54eb-46af-a96d-395d749f9ae5 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:32.388155 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 382/780] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:32.389459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d619d2-54eb-46af-a96d-395d749f9ae5 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:32.443080 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:32.518891 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9c724-266a-45d6-b392-5ef7f96db901 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:32.532162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d619d2-54eb-46af-a96d-395d749f9ae5 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:32.564461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce095892-4e81-492a-bf48-5395d68bb343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:32.565564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce095892-4e81-492a-bf48-5395d68bb343 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:32.566064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce095892-4e81-492a-bf48-5395d68bb343 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:32.566856 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce095892-4e81-492a-bf48-5395d68bb343 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:32.567956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce095892-4e81-492a-bf48-5395d68bb343 None admin] RBAC: Adding query filter params (interface=public, service_id=a938306e6416430aa4dc5af8e5d12bb5, region_id=RegionOne) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:32.568638 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce095892-4e81-492a-bf48-5395d68bb343 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:32.570397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce095892-4e81-492a-bf48-5395d68bb343 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:32.582153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 399/781] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Jul 14 17:38:32 2022] GET /identity/v3/endpoints?service_id=a938306e6416430aa4dc5af8e5d12bb5&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:32.583388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519222b1-7d3c-48b7-aeca-528a8017076c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:32.583705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce095892-4e81-492a-bf48-5395d68bb343 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:32.584163 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519222b1-7d3c-48b7-aeca-528a8017076c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:32.584451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519222b1-7d3c-48b7-aeca-528a8017076c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:32.584746 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519222b1-7d3c-48b7-aeca-528a8017076c None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:32.585502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519222b1-7d3c-48b7-aeca-528a8017076c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:32.585810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519222b1-7d3c-48b7-aeca-528a8017076c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:32.587530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519222b1-7d3c-48b7-aeca-528a8017076c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:32.604028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 383/782] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:38:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:32.605555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519222b1-7d3c-48b7-aeca-528a8017076c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:32.681658 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:32.726971 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce095892-4e81-492a-bf48-5395d68bb343 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:32.755133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:32.765018 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519222b1-7d3c-48b7-aeca-528a8017076c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:32.788679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6933cc-a23c-402d-a35e-c4238670144b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:32.790355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6933cc-a23c-402d-a35e-c4238670144b None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:32.791131 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6933cc-a23c-402d-a35e-c4238670144b None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:32.791857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6933cc-a23c-402d-a35e-c4238670144b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:32.793443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6933cc-a23c-402d-a35e-c4238670144b None admin] RBAC: Adding query filter params (role.id=2db2bc363aa74930932c823dc291d7f3, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=0cd1747822c3493e8aec150a12409fbd) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:32.794191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6933cc-a23c-402d-a35e-c4238670144b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:32.796307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6933cc-a23c-402d-a35e-c4238670144b None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:32.812161 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 400/783] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:32 2022] GET /identity/v3/role_assignments?user.id=0cd1747822c3493e8aec150a12409fbd&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=2db2bc363aa74930932c823dc291d7f3 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:32.813637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6933cc-a23c-402d-a35e-c4238670144b None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:32.820565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204a8d6-dde6-44dd-b986-765f165cc417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:32.821980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204a8d6-dde6-44dd-b986-765f165cc417 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:32.821980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204a8d6-dde6-44dd-b986-765f165cc417 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:32.822274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204a8d6-dde6-44dd-b986-765f165cc417 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:32.823156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204a8d6-dde6-44dd-b986-765f165cc417 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:32.824740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204a8d6-dde6-44dd-b986-765f165cc417 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:32.834479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 384/784] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Jul 14 17:38:32 2022] GET /identity/v3/services => generated 772 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:33.020786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6933cc-a23c-402d-a35e-c4238670144b None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:33.069502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d6d67-855d-4e59-9934-d3896b7a280d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:33.070313 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d6d67-855d-4e59-9934-d3896b7a280d None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:33.070706 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d6d67-855d-4e59-9934-d3896b7a280d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:33.070987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d6d67-855d-4e59-9934-d3896b7a280d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:33.095390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d6d67-855d-4e59-9934-d3896b7a280d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:33.095390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d6d67-855d-4e59-9934-d3896b7a280d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-745d6d67-855d-4e59-9934-d3896b7a280d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:33.118431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:33.120275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:33.121188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:33.121188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:33.121188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:33.121188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:33.121188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 401/785] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:33.124703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204a8d6-dde6-44dd-b986-765f165cc417 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:33.274015 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204a8d6-dde6-44dd-b986-765f165cc417 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:33.290157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:33.316761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b34c0-ad74-4f2a-a7c0-3868309d7474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:33.317437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b34c0-ad74-4f2a-a7c0-3868309d7474 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:33.317725 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b34c0-ad74-4f2a-a7c0-3868309d7474 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:33.318002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b34c0-ad74-4f2a-a7c0-3868309d7474 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:33.318737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b34c0-ad74-4f2a-a7c0-3868309d7474 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:33.319047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b34c0-ad74-4f2a-a7c0-3868309d7474 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:33.320483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b34c0-ad74-4f2a-a7c0-3868309d7474 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:33.334596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 385/786] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:33.340084 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d6d67-855d-4e59-9934-d3896b7a280d None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:33.382580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:33.474542 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d6d67-855d-4e59-9934-d3896b7a280d None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:33.523764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e783d6-d059-4c8c-bb9d-426c1bc8a7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:33.524677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e783d6-d059-4c8c-bb9d-426c1bc8a7c2 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:33.524901 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e783d6-d059-4c8c-bb9d-426c1bc8a7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:33.525178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e783d6-d059-4c8c-bb9d-426c1bc8a7c2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:33.533913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e783d6-d059-4c8c-bb9d-426c1bc8a7c2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:33.535392 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e783d6-d059-4c8c-bb9d-426c1bc8a7c2 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-44e783d6-d059-4c8c-bb9d-426c1bc8a7c2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:33.551142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:33.552360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:33.553949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:33.553949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:33.553949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:33.553949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:33.553949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:33.553949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:33.553949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:33.553949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:33.553949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:38:33.553949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:33.553949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 402/787] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Jul 14 17:38:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:33.558804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b34c0-ad74-4f2a-a7c0-3868309d7474 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:33.722914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b34c0-ad74-4f2a-a7c0-3868309d7474 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:33.766165 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934055f-fa13-473f-be77-86e0686790a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:33.766871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934055f-fa13-473f-be77-86e0686790a6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:33.767156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934055f-fa13-473f-be77-86e0686790a6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:33.767435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934055f-fa13-473f-be77-86e0686790a6 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:33.768309 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934055f-fa13-473f-be77-86e0686790a6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:33.768625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934055f-fa13-473f-be77-86e0686790a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:33.770078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934055f-fa13-473f-be77-86e0686790a6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:33.790701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 386/788] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Jul 14 17:38:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:33.796434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e783d6-d059-4c8c-bb9d-426c1bc8a7c2 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:33.909831 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:33.919600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e783d6-d059-4c8c-bb9d-426c1bc8a7c2 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:33.957765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ab6d0-a5c3-4bb7-8a96-347f6a81a348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:33.958894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ab6d0-a5c3-4bb7-8a96-347f6a81a348 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:33.959395 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ab6d0-a5c3-4bb7-8a96-347f6a81a348 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:33.959872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ab6d0-a5c3-4bb7-8a96-347f6a81a348 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:34.044218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ab6d0-a5c3-4bb7-8a96-347f6a81a348 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:34.046045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8ab6d0-a5c3-4bb7-8a96-347f6a81a348 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:34.152714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:34.163038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 403/789] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:38:34.168511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934055f-fa13-473f-be77-86e0686790a6 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:34.223886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:34.308862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934055f-fa13-473f-be77-86e0686790a6 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:34.346728 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0239f-93d1-4eb4-9036-f2d2d1f361f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:34.347601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0239f-93d1-4eb4-9036-f2d2d1f361f3 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:34.347898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0239f-93d1-4eb4-9036-f2d2d1f361f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:34.348239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0239f-93d1-4eb4-9036-f2d2d1f361f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:34.360642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0239f-93d1-4eb4-9036-f2d2d1f361f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:34.362156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0239f-93d1-4eb4-9036-f2d2d1f361f3 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-4ca0239f-93d1-4eb4-9036-f2d2d1f361f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:34.383437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:34.384425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:34.385324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:34.385324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:34.385324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:34.385324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:34.385324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 387/790] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:34.397168 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ab6d0-a5c3-4bb7-8a96-347f6a81a348 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:34.515987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ab6d0-a5c3-4bb7-8a96-347f6a81a348 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:34.524294 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8959b-639f-45c1-813b-144559691406 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:34.524916 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8959b-639f-45c1-813b-144559691406 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:34.525400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de8959b-639f-45c1-813b-144559691406 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:34.527318 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 388/791] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:34.533994 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57c13c-6918-47ab-9fec-ecc6c1bf466b None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:34.534724 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57c13c-6918-47ab-9fec-ecc6c1bf466b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:34.535213 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57c13c-6918-47ab-9fec-ecc6c1bf466b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:34.554981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0994c53-e211-42af-b47f-0a2e4222b744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:34.555656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0994c53-e211-42af-b47f-0a2e4222b744 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:34.555974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0994c53-e211-42af-b47f-0a2e4222b744 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:34.556237 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0994c53-e211-42af-b47f-0a2e4222b744 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:34.556900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-8e57c13c-6918-47ab-9fec-ecc6c1bf466b None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:34.557010 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0994c53-e211-42af-b47f-0a2e4222b744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:34.557257 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0994c53-e211-42af-b47f-0a2e4222b744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:34.558614 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0994c53-e211-42af-b47f-0a2e4222b744 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:34.558850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8e57c13c-6918-47ab-9fec-ecc6c1bf466b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:34.575290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 404/792] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:34.576539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0994c53-e211-42af-b47f-0a2e4222b744 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:34.669678 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 389/793] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:38:34.671020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe3966-42f3-4338-a66e-e3f74a2389d8 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:34.671310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe3966-42f3-4338-a66e-e3f74a2389d8 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:34.671587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe3966-42f3-4338-a66e-e3f74a2389d8 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:34.675478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 390/794] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:34.675667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:34.686965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b65816-6aac-454d-a73a-f0f1d7230b71 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:34.687559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b65816-6aac-454d-a73a-f0f1d7230b71 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:34.688085 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b65816-6aac-454d-a73a-f0f1d7230b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:34.710243 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-30b65816-6aac-454d-a73a-f0f1d7230b71 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:34.712032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-30b65816-6aac-454d-a73a-f0f1d7230b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:34.715751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:34.738765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0994c53-e211-42af-b47f-0a2e4222b744 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:34.774992 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914706c3-c84e-4caa-9508-b7407833776a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:34.776246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914706c3-c84e-4caa-9508-b7407833776a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:34.776861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914706c3-c84e-4caa-9508-b7407833776a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:34.777470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914706c3-c84e-4caa-9508-b7407833776a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:34.785644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914706c3-c84e-4caa-9508-b7407833776a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:34.787397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914706c3-c84e-4caa-9508-b7407833776a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:34.795273 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 391/795] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:38:34.796211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b65816-6aac-454d-a73a-f0f1d7230b71 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-914706c3-c84e-4caa-9508-b7407833776a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:34.804011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:34.804916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:34.805863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:34.805863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:34.805863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:34.805863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:34.805863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:34.805863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:34.805863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:34.805863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:34.808503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 405/796] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:34.814439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914706c3-c84e-4caa-9508-b7407833776a None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:34.928707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914706c3-c84e-4caa-9508-b7407833776a None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:34.940231 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b65816-6aac-454d-a73a-f0f1d7230b71 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:34.943351 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:34.968226 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9a0fb-573c-490e-be9b-b7f324aa7803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:34.969060 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9a0fb-573c-490e-be9b-b7f324aa7803 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:34.969442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9a0fb-573c-490e-be9b-b7f324aa7803 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:34.969814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc9a0fb-573c-490e-be9b-b7f324aa7803 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:34.970738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9a0fb-573c-490e-be9b-b7f324aa7803 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:34.971145 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9a0fb-573c-490e-be9b-b7f324aa7803 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:34.973302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc9a0fb-573c-490e-be9b-b7f324aa7803 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:34.991471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 406/797] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:34.991471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9a0fb-573c-490e-be9b-b7f324aa7803 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:34.992072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e3251-9841-4012-8dc6-8bfc9bacddd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:34.994121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e3251-9841-4012-8dc6-8bfc9bacddd0 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:34.994574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e3251-9841-4012-8dc6-8bfc9bacddd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:34.995002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e3251-9841-4012-8dc6-8bfc9bacddd0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:35.006292 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:35.011937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223e3251-9841-4012-8dc6-8bfc9bacddd0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-223e3251-9841-4012-8dc6-8bfc9bacddd0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:35.023735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:35.025324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:35.026370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:35.026370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:35.026370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:35.026370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:35.026370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:35.026370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:35.026370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:35.026370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:35.026370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:35.026370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:35.026370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:38:35.026370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:35.034246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 392/798] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:38:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:35.034246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e3251-9841-4012-8dc6-8bfc9bacddd0 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:35.123638 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc9a0fb-573c-490e-be9b-b7f324aa7803 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:35.165722 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e3251-9841-4012-8dc6-8bfc9bacddd0 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:35.171799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e575d-1c5e-4955-b4fd-7965d20d30e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:35.173174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e575d-1c5e-4955-b4fd-7965d20d30e6 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:35.173610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e575d-1c5e-4955-b4fd-7965d20d30e6 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:35.174608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e575d-1c5e-4955-b4fd-7965d20d30e6 None admin] PATH_INFO: `/v3/services/network` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:35.175466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e575d-1c5e-4955-b4fd-7965d20d30e6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:35.177510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0e575d-1c5e-4955-b4fd-7965d20d30e6 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-3a0e575d-1c5e-4955-b4fd-7965d20d30e6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:35.194864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:35.195875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:35.196823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:35.196823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:35.196823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:35.196823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:35.196823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:35.196823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:35.196823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:35.196823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:35.196823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:35.196823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:35.196823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:35.196823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:35.196823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 17:38:35.196823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:35.203010 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 407/799] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:34 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:35.205103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e575d-1c5e-4955-b4fd-7965d20d30e6 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:35.211967 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d3b30-35a0-41f0-a463-eafed415237d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:35.212621 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d3b30-35a0-41f0-a463-eafed415237d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:35.212907 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d3b30-35a0-41f0-a463-eafed415237d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:35.213183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d3b30-35a0-41f0-a463-eafed415237d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:35.214024 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d3b30-35a0-41f0-a463-eafed415237d None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=6423f54152064b909d03c0908d636993) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:35.214344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d3b30-35a0-41f0-a463-eafed415237d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:35.215882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d3b30-35a0-41f0-a463-eafed415237d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:35.228230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 393/800] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:35 2022] GET /identity/v3/role_assignments?user.id=6423f54152064b909d03c0908d636993&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:35.228230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d3b30-35a0-41f0-a463-eafed415237d None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:35.283182 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:35.367817 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:35.423423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e575d-1c5e-4955-b4fd-7965d20d30e6 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:35.440458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d3b30-35a0-41f0-a463-eafed415237d None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:35.467464 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718ab169-519a-4423-887d-e14d30574682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:35.468521 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ab169-519a-4423-887d-e14d30574682 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:35.469029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ab169-519a-4423-887d-e14d30574682 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:35.469487 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718ab169-519a-4423-887d-e14d30574682 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:35.470444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ab169-519a-4423-887d-e14d30574682 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:35.470952 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ab169-519a-4423-887d-e14d30574682 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:35.472678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718ab169-519a-4423-887d-e14d30574682 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:35.483653 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 408/801] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:38:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:35.485431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc5faf-d686-4852-92d2-869bfd7aacf8 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:35.485551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7740b8f9-caa1-45f6-b517-1de48e2ef4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:35.486007 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc5faf-d686-4852-92d2-869bfd7aacf8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:35.486430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc5faf-d686-4852-92d2-869bfd7aacf8 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:35.486593 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740b8f9-caa1-45f6-b517-1de48e2ef4eb None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:35.486911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740b8f9-caa1-45f6-b517-1de48e2ef4eb None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:35.487290 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7740b8f9-caa1-45f6-b517-1de48e2ef4eb None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:35.488536 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740b8f9-caa1-45f6-b517-1de48e2ef4eb None admin] RBAC: Adding query filter params (name=network) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:35.488888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 409/802] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:35.488999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740b8f9-caa1-45f6-b517-1de48e2ef4eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:35.489842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc5faf-d686-4852-92d2-869bfd7aacf8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:35.490600 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7740b8f9-caa1-45f6-b517-1de48e2ef4eb None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:35.501774 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 394/803] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Jul 14 17:38:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:35.503295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e535da-db88-45f6-8f2c-b8845e2b37ea None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:35.503733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e535da-db88-45f6-8f2c-b8845e2b37ea None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:35.504192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e535da-db88-45f6-8f2c-b8845e2b37ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:35.522644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-a8e535da-db88-45f6-8f2c-b8845e2b37ea None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:35.523832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a8e535da-db88-45f6-8f2c-b8845e2b37ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:35.598748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 395/804] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:35.599609 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e535da-db88-45f6-8f2c-b8845e2b37ea None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:35.606353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:35.613076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc5faf-d686-4852-92d2-869bfd7aacf8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:35.654090 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345f85f-dbb9-4916-9f76-03d1d00a57a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:35.654882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345f85f-dbb9-4916-9f76-03d1d00a57a7 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:35.655178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345f85f-dbb9-4916-9f76-03d1d00a57a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:35.655442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345f85f-dbb9-4916-9f76-03d1d00a57a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:35.659840 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:35.667119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345f85f-dbb9-4916-9f76-03d1d00a57a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:35.668610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4345f85f-dbb9-4916-9f76-03d1d00a57a7 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-4345f85f-dbb9-4916-9f76-03d1d00a57a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:35.686882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:35.688048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:35.693991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:35.693991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:35.693991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:35.693991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:35.693991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 410/805] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:35.693991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa9649-eb5f-4c1a-8f1d-2cf5c99b43b8 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:35.693991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa9649-eb5f-4c1a-8f1d-2cf5c99b43b8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:35.693991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa9649-eb5f-4c1a-8f1d-2cf5c99b43b8 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:35.693991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 411/806] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:35.694492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa9649-eb5f-4c1a-8f1d-2cf5c99b43b8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:35.716711 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e535da-db88-45f6-8f2c-b8845e2b37ea None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:35.755130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8792805-a203-47ce-b857-688201c0d2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:35.755727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8792805-a203-47ce-b857-688201c0d2ff None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:35.756048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8792805-a203-47ce-b857-688201c0d2ff None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:35.756330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8792805-a203-47ce-b857-688201c0d2ff None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:35.757040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8792805-a203-47ce-b857-688201c0d2ff None admin] RBAC: Adding query filter params (type=network) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:35.757351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8792805-a203-47ce-b857-688201c0d2ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:35.758845 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8792805-a203-47ce-b857-688201c0d2ff None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:35.764516 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 396/807] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Jul 14 17:38:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:35.765348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8792805-a203-47ce-b857-688201c0d2ff None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:35.811901 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa9649-eb5f-4c1a-8f1d-2cf5c99b43b8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:35.846318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cc3bf6-904c-4bbd-944c-e50e88577302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:35.847137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3bf6-904c-4bbd-944c-e50e88577302 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:35.847425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3bf6-904c-4bbd-944c-e50e88577302 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:35.847703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cc3bf6-904c-4bbd-944c-e50e88577302 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:35.848397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc3bf6-904c-4bbd-944c-e50e88577302 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:35.849769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cc3bf6-904c-4bbd-944c-e50e88577302 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-d2cc3bf6-904c-4bbd-944c-e50e88577302 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:35.862860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:35.863805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:35.864723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 412/808] 10.222.0.64 () {56 vars in 1129 bytes} [Thu Jul 14 17:38:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:35.865571 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf8bdf-aeec-4b4e-9a5b-6f8c0551c4e8 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:35.866301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf8bdf-aeec-4b4e-9a5b-6f8c0551c4e8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:35.866301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf8bdf-aeec-4b4e-9a5b-6f8c0551c4e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:35.877903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8792805-a203-47ce-b857-688201c0d2ff None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:35.884739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4ebf8bdf-aeec-4b4e-9a5b-6f8c0551c4e8 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:35.885946 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4ebf8bdf-aeec-4b4e-9a5b-6f8c0551c4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:35.896684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:35.902280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:35.917352 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfe812-4e5b-4a92-8eb3-cac333787a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:35.918029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfe812-4e5b-4a92-8eb3-cac333787a7b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:35.918316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfe812-4e5b-4a92-8eb3-cac333787a7b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:35.918598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cfe812-4e5b-4a92-8eb3-cac333787a7b None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:35.919967 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cfe812-4e5b-4a92-8eb3-cac333787a7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:35.920346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cfe812-4e5b-4a92-8eb3-cac333787a7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:35.921757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cfe812-4e5b-4a92-8eb3-cac333787a7b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:35.934016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 397/809] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:35.934702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfe812-4e5b-4a92-8eb3-cac333787a7b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:35.971966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 413/810] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:35.971966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf8bdf-aeec-4b4e-9a5b-6f8c0551c4e8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:36.059839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cfe812-4e5b-4a92-8eb3-cac333787a7b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:36.079427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf8bdf-aeec-4b4e-9a5b-6f8c0551c4e8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:36.098186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed473d32-85a9-4e13-91ff-31df4f5936cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:36.098855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed473d32-85a9-4e13-91ff-31df4f5936cf None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:36.099140 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed473d32-85a9-4e13-91ff-31df4f5936cf None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:36.099417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed473d32-85a9-4e13-91ff-31df4f5936cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:36.100185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed473d32-85a9-4e13-91ff-31df4f5936cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:36.101564 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed473d32-85a9-4e13-91ff-31df4f5936cf None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:36.117247 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fe0ea-29e4-4576-8874-d5451280f3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:36.117932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fe0ea-29e4-4576-8874-d5451280f3a5 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:36.118218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fe0ea-29e4-4576-8874-d5451280f3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:36.118526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fe0ea-29e4-4576-8874-d5451280f3a5 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:36.119299 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fe0ea-29e4-4576-8874-d5451280f3a5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:36.119633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fe0ea-29e4-4576-8874-d5451280f3a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:36.121412 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6fe0ea-29e4-4576-8874-d5451280f3a5 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:36.127250 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 398/811] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Jul 14 17:38:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:38:36.127513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 414/812] 10.222.0.64 () {56 vars in 1124 bytes} [Thu Jul 14 17:38:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:36.137103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fe0ea-29e4-4576-8874-d5451280f3a5 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:36.137339 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed473d32-85a9-4e13-91ff-31df4f5936cf None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:36.154239 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:36.187710 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:36.273499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed473d32-85a9-4e13-91ff-31df4f5936cf None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:36.299189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6fe0ea-29e4-4576-8874-d5451280f3a5 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:36.324753 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdedd4-1378-4ead-9c7a-9562d8c93e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:36.325599 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdedd4-1378-4ead-9c7a-9562d8c93e97 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:36.325892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdedd4-1378-4ead-9c7a-9562d8c93e97 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:36.326173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdedd4-1378-4ead-9c7a-9562d8c93e97 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:36.340138 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdedd4-1378-4ead-9c7a-9562d8c93e97 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:36.341653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbdedd4-1378-4ead-9c7a-9562d8c93e97 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:36.342453 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93777b4-713c-477c-b8a6-2c9441ec70eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:36.343430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93777b4-713c-477c-b8a6-2c9441ec70eb None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:36.343866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93777b4-713c-477c-b8a6-2c9441ec70eb None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:36.344317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93777b4-713c-477c-b8a6-2c9441ec70eb None admin] PATH_INFO: `/v3/roles/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:36.358226 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93777b4-713c-477c-b8a6-2c9441ec70eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-ffbdedd4-1378-4ead-9c7a-9562d8c93e97 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:36.360945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:36.362117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:36.363307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:36.363307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:36.363307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:36.363307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:36.363307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:36.363307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:36.363307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:36.363307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:36.363307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 17:38:36.363307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:36.363307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 399/813] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:38:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:36.363307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdedd4-1378-4ead-9c7a-9562d8c93e97 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-f93777b4-713c-477c-b8a6-2c9441ec70eb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:36.371890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:36.373948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:36.375003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:36.375003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:36.375003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:36.375003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:36.375003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:36.375003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:36.375003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:36.375003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:36.375003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:36.375003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:36.375003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:36.375003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:36.377727 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 415/814] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:36.379324 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93777b4-713c-477c-b8a6-2c9441ec70eb None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:36.456217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:36.476228 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:36.484974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbdedd4-1378-4ead-9c7a-9562d8c93e97 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:36.498307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93777b4-713c-477c-b8a6-2c9441ec70eb None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:36.525861 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99c409-90ea-4ff6-9515-e4394c306317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:36.526572 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99c409-90ea-4ff6-9515-e4394c306317 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:36.526867 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99c409-90ea-4ff6-9515-e4394c306317 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:36.527163 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99c409-90ea-4ff6-9515-e4394c306317 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:36.527977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99c409-90ea-4ff6-9515-e4394c306317 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:36.528297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99c409-90ea-4ff6-9515-e4394c306317 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:36.530322 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a99c409-90ea-4ff6-9515-e4394c306317 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:36.535653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 400/815] 10.222.0.64 () {56 vars in 1124 bytes} [Thu Jul 14 17:38:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:36.536619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99c409-90ea-4ff6-9515-e4394c306317 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:36.547312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae6582-4e41-41d7-86b4-2f6f5829c322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:36.548026 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae6582-4e41-41d7-86b4-2f6f5829c322 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:36.548317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae6582-4e41-41d7-86b4-2f6f5829c322 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:36.548598 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae6582-4e41-41d7-86b4-2f6f5829c322 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:36.549457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae6582-4e41-41d7-86b4-2f6f5829c322 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:36.549769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae6582-4e41-41d7-86b4-2f6f5829c322 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:36.551203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae6582-4e41-41d7-86b4-2f6f5829c322 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:36.572385 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 416/816] 10.222.0.64 () {56 vars in 1135 bytes} [Thu Jul 14 17:38:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:36.573055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae6582-4e41-41d7-86b4-2f6f5829c322 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:36.661261 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a99c409-90ea-4ff6-9515-e4394c306317 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:36.696938 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ec5aa-71de-42fe-a64f-6bf66d76b31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:36.697638 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ec5aa-71de-42fe-a64f-6bf66d76b31e None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:36.697917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ec5aa-71de-42fe-a64f-6bf66d76b31e None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:36.698193 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ec5aa-71de-42fe-a64f-6bf66d76b31e None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:36.698972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ec5aa-71de-42fe-a64f-6bf66d76b31e None admin] RBAC: Adding query filter params (name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:36.699277 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ec5aa-71de-42fe-a64f-6bf66d76b31e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:36.700543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae6582-4e41-41d7-86b4-2f6f5829c322 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:36.700797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ec5aa-71de-42fe-a64f-6bf66d76b31e None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:36.709497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 401/817] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:36.710476 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ec5aa-71de-42fe-a64f-6bf66d76b31e None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:36.728929 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:36.738524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:36.741897 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6119be-0001-4ba7-8e2b-32388d0d9ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:36.742565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6119be-0001-4ba7-8e2b-32388d0d9ad8 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:36.742847 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6119be-0001-4ba7-8e2b-32388d0d9ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:36.743119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6119be-0001-4ba7-8e2b-32388d0d9ad8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:36.743859 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6119be-0001-4ba7-8e2b-32388d0d9ad8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:36.745310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6119be-0001-4ba7-8e2b-32388d0d9ad8 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:36.775952 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 417/818] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Jul 14 17:38:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:38:36.775952 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6119be-0001-4ba7-8e2b-32388d0d9ad8 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:36.857660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ec5aa-71de-42fe-a64f-6bf66d76b31e None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:36.901300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6119be-0001-4ba7-8e2b-32388d0d9ad8 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:36.911510 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22867b2b-cc12-4c0b-a2e6-d25e787e7393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:36.912960 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22867b2b-cc12-4c0b-a2e6-d25e787e7393 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:36.913333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22867b2b-cc12-4c0b-a2e6-d25e787e7393 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:36.913689 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22867b2b-cc12-4c0b-a2e6-d25e787e7393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:36.930224 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22867b2b-cc12-4c0b-a2e6-d25e787e7393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:36.932622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22867b2b-cc12-4c0b-a2e6-d25e787e7393 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:36.942154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9df74-d6b3-4922-8e5f-e818fa3b5a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:36.943121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9df74-d6b3-4922-8e5f-e818fa3b5a36 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:36.943547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9df74-d6b3-4922-8e5f-e818fa3b5a36 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:36.943946 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b9df74-d6b3-4922-8e5f-e818fa3b5a36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-22867b2b-cc12-4c0b-a2e6-d25e787e7393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:36.957972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:36.959050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:36.960295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:36.960295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:36.960295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:36.960295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:36.960573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b9df74-d6b3-4922-8e5f-e818fa3b5a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:36.960667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 402/819] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:36.961850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b9df74-d6b3-4922-8e5f-e818fa3b5a36 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:36.966972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22867b2b-cc12-4c0b-a2e6-d25e787e7393 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-73b9df74-d6b3-4922-8e5f-e818fa3b5a36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:36.981405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:36.982941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:36.983829 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:36.983829 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:36.983829 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:36.983829 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:36.986772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 418/820] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:36.989084 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9df74-d6b3-4922-8e5f-e818fa3b5a36 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:37.025192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:37.059414 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:37.094943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22867b2b-cc12-4c0b-a2e6-d25e787e7393 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:37.138550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b9df74-d6b3-4922-8e5f-e818fa3b5a36 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:37.141175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf41943-0cea-492d-a7e4-6b254d412402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:37.141848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf41943-0cea-492d-a7e4-6b254d412402 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:37.142134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf41943-0cea-492d-a7e4-6b254d412402 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:37.142412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf41943-0cea-492d-a7e4-6b254d412402 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:37.143135 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf41943-0cea-492d-a7e4-6b254d412402 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:37.143445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf41943-0cea-492d-a7e4-6b254d412402 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:37.145634 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf41943-0cea-492d-a7e4-6b254d412402 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:37.158349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 403/821] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:37.163918 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf41943-0cea-492d-a7e4-6b254d412402 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:37.189839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bfe98-66c5-47a9-82ea-56fdd0935a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:37.191029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bfe98-66c5-47a9-82ea-56fdd0935a4e None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:37.191978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bfe98-66c5-47a9-82ea-56fdd0935a4e None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:37.192689 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0bfe98-66c5-47a9-82ea-56fdd0935a4e None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:37.193850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bfe98-66c5-47a9-82ea-56fdd0935a4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:37.194711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bfe98-66c5-47a9-82ea-56fdd0935a4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:37.196810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0bfe98-66c5-47a9-82ea-56fdd0935a4e None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:37.214097 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 419/822] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:37.219585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bfe98-66c5-47a9-82ea-56fdd0935a4e None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:37.278175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf41943-0cea-492d-a7e4-6b254d412402 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:37.312980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:37.318616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f10cb-5f7d-4ea9-bad3-c7a48e501547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:37.319960 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f10cb-5f7d-4ea9-bad3-c7a48e501547 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:37.320461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f10cb-5f7d-4ea9-bad3-c7a48e501547 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:37.321160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f10cb-5f7d-4ea9-bad3-c7a48e501547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:37.329252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f10cb-5f7d-4ea9-bad3-c7a48e501547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:37.330882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f10cb-5f7d-4ea9-bad3-c7a48e501547 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:37.342412 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-f64f10cb-5f7d-4ea9-bad3-c7a48e501547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:37.345981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:37.347074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:37.348152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:37.348152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:37.348152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:37.348152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:37.348152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:37.348152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:37.348152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:37.348152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:37.354178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 404/823] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:37.355553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f10cb-5f7d-4ea9-bad3-c7a48e501547 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:37.358015 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0bfe98-66c5-47a9-82ea-56fdd0935a4e None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:37.405788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44afd0e-8d8d-4242-a435-e64acf6b9815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:37.406724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44afd0e-8d8d-4242-a435-e64acf6b9815 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:37.407463 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44afd0e-8d8d-4242-a435-e64acf6b9815 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:37.407812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44afd0e-8d8d-4242-a435-e64acf6b9815 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:37.418205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44afd0e-8d8d-4242-a435-e64acf6b9815 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:37.420560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44afd0e-8d8d-4242-a435-e64acf6b9815 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-d44afd0e-8d8d-4242-a435-e64acf6b9815 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:37.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:37.442877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:37.443803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:37.443803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:37.443803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:37.443803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:37.443803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:37.443803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:37.443803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:37.443803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:37.443803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 17:38:37.443803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:37.448101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 420/824] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Jul 14 17:38:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:37.451643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44afd0e-8d8d-4242-a435-e64acf6b9815 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:37.486380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f10cb-5f7d-4ea9-bad3-c7a48e501547 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:37.530439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dad274e-b3e1-47ab-8b5f-2769de9ebdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:37.531960 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad274e-b3e1-47ab-8b5f-2769de9ebdea None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:37.532454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad274e-b3e1-47ab-8b5f-2769de9ebdea None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:37.532924 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dad274e-b3e1-47ab-8b5f-2769de9ebdea None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:37.534393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad274e-b3e1-47ab-8b5f-2769de9ebdea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:37.534891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad274e-b3e1-47ab-8b5f-2769de9ebdea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:37.536907 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dad274e-b3e1-47ab-8b5f-2769de9ebdea None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:37.556113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 405/825] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:37.556113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d884b57-552c-42a0-9891-cb0b6aec5e96 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:37.556385 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44afd0e-8d8d-4242-a435-e64acf6b9815 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:37.556462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d884b57-552c-42a0-9891-cb0b6aec5e96 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:37.556462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d884b57-552c-42a0-9891-cb0b6aec5e96 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:37.556462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 406/826] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:37.559697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d884b57-552c-42a0-9891-cb0b6aec5e96 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:37.592345 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05ea13-f9d5-4d91-9fb3-216689144863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:37.593017 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05ea13-f9d5-4d91-9fb3-216689144863 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:37.593305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05ea13-f9d5-4d91-9fb3-216689144863 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:37.593589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05ea13-f9d5-4d91-9fb3-216689144863 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:37.594427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05ea13-f9d5-4d91-9fb3-216689144863 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:37.594743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05ea13-f9d5-4d91-9fb3-216689144863 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:37.596203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05ea13-f9d5-4d91-9fb3-216689144863 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:37.614331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 421/827] 10.222.0.64 () {56 vars in 1149 bytes} [Thu Jul 14 17:38:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:37.615632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ec5e5-08dc-4d07-9412-7e11767c9c4c None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:37.615942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ec5e5-08dc-4d07-9412-7e11767c9c4c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:37.616228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ec5e5-08dc-4d07-9412-7e11767c9c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:37.624108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:37.633439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-fd7ec5e5-08dc-4d07-9412-7e11767c9c4c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:37.634666 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-fd7ec5e5-08dc-4d07-9412-7e11767c9c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:37.636678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:37.678043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d884b57-552c-42a0-9891-cb0b6aec5e96 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:37.711456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 422/828] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:37.712379 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ec5e5-08dc-4d07-9412-7e11767c9c4c None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:37.723775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca6a12-7e1b-409d-b7a1-ffa904cae63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:37.724775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca6a12-7e1b-409d-b7a1-ffa904cae63d None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:37.725108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca6a12-7e1b-409d-b7a1-ffa904cae63d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:37.725385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aca6a12-7e1b-409d-b7a1-ffa904cae63d None admin] PATH_INFO: `/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/97f15306528e415baa5c69f12f22c75e/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:37.728220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca6a12-7e1b-409d-b7a1-ffa904cae63d None admin] RBAC: Authorizing `identity:create_grant(project_id=de62d2fb17f846f2bce5bef108353fd8, user_id=97f15306528e415baa5c69f12f22c75e, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:37.730222 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aca6a12-7e1b-409d-b7a1-ffa904cae63d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:37.744921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 407/829] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:38:37 2022] PUT /identity/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/97f15306528e415baa5c69f12f22c75e/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:38:37.756990 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca6a12-7e1b-409d-b7a1-ffa904cae63d None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:37.847458 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ec5e5-08dc-4d07-9412-7e11767c9c4c None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:37.890001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4a9c2-f7bc-42a0-96b2-a8ff9152acd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:37.890944 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a9c2-f7bc-42a0-96b2-a8ff9152acd0 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:37.891241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a9c2-f7bc-42a0-96b2-a8ff9152acd0 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:37.891524 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4a9c2-f7bc-42a0-96b2-a8ff9152acd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:37.896278 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aca6a12-7e1b-409d-b7a1-ffa904cae63d None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:37.899704 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:37.904306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4a9c2-f7bc-42a0-96b2-a8ff9152acd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:37.905881 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4a9c2-f7bc-42a0-96b2-a8ff9152acd0 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:37.915841 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-cdf4a9c2-f7bc-42a0-96b2-a8ff9152acd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:37.929532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:37.930526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:37.931427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:37.931427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:37.931427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:37.931427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:37.931427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 423/830] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:37.938872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4a9c2-f7bc-42a0-96b2-a8ff9152acd0 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:37.952357 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac8bdd-5bed-4205-a9b9-402bcdda3eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:37.953227 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac8bdd-5bed-4205-a9b9-402bcdda3eac None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:37.953520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac8bdd-5bed-4205-a9b9-402bcdda3eac None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:37.953803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ac8bdd-5bed-4205-a9b9-402bcdda3eac None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:37.954567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac8bdd-5bed-4205-a9b9-402bcdda3eac None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:37.956002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ac8bdd-5bed-4205-a9b9-402bcdda3eac None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-58ac8bdd-5bed-4205-a9b9-402bcdda3eac None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:37.969394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:37.970429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:37.971281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 408/831] 10.222.0.64 () {56 vars in 1132 bytes} [Thu Jul 14 17:38:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:37.977246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac8bdd-5bed-4205-a9b9-402bcdda3eac None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:38.084428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4a9c2-f7bc-42a0-96b2-a8ff9152acd0 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:38.109540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ac8bdd-5bed-4205-a9b9-402bcdda3eac None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:38.133123 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de610088-ee98-4d3b-8272-91e32c90bb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:38.134138 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de610088-ee98-4d3b-8272-91e32c90bb1b None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:38.134429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de610088-ee98-4d3b-8272-91e32c90bb1b None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:38.134717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de610088-ee98-4d3b-8272-91e32c90bb1b None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:38.135483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de610088-ee98-4d3b-8272-91e32c90bb1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:38.135796 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de610088-ee98-4d3b-8272-91e32c90bb1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:38.137310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de610088-ee98-4d3b-8272-91e32c90bb1b None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:38.156076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704199f-05ea-4299-ae05-d8f14ec62287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:38.157024 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704199f-05ea-4299-ae05-d8f14ec62287 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:38.157489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704199f-05ea-4299-ae05-d8f14ec62287 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:38.157964 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704199f-05ea-4299-ae05-d8f14ec62287 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:38.158930 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704199f-05ea-4299-ae05-d8f14ec62287 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:38.159070 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 424/832] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:38.159587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704199f-05ea-4299-ae05-d8f14ec62287 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:38.160045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de610088-ee98-4d3b-8272-91e32c90bb1b None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:38.161496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704199f-05ea-4299-ae05-d8f14ec62287 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:38.167584 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 409/833] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Jul 14 17:38:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:38.176226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704199f-05ea-4299-ae05-d8f14ec62287 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:38.228256 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:38.260306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:38.298268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704199f-05ea-4299-ae05-d8f14ec62287 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:38.300575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de610088-ee98-4d3b-8272-91e32c90bb1b None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:38.338391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fe904-42c7-44f0-a0d3-ee1618dcd118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:38.339392 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fe904-42c7-44f0-a0d3-ee1618dcd118 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:38.339679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fe904-42c7-44f0-a0d3-ee1618dcd118 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:38.340068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1fe904-42c7-44f0-a0d3-ee1618dcd118 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:38.340431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6244ae-cfa9-457a-976e-5a78e6c0ed20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:38.341100 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6244ae-cfa9-457a-976e-5a78e6c0ed20 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:38.341903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6244ae-cfa9-457a-976e-5a78e6c0ed20 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:38.342181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6244ae-cfa9-457a-976e-5a78e6c0ed20 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:38.343104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6244ae-cfa9-457a-976e-5a78e6c0ed20 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:38.343412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6244ae-cfa9-457a-976e-5a78e6c0ed20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:38.344831 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6244ae-cfa9-457a-976e-5a78e6c0ed20 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:38.356173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 410/834] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Jul 14 17:38:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:38.358071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fe904-42c7-44f0-a0d3-ee1618dcd118 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:38.359745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1fe904-42c7-44f0-a0d3-ee1618dcd118 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-3c1fe904-42c7-44f0-a0d3-ee1618dcd118 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:38.376251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:38.377196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:38.378688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:38.378688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:38.378688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:38.378688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:38.378688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:38.378688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:38.378688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:38.378688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:38.378688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 425/835] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:38.383413 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6244ae-cfa9-457a-976e-5a78e6c0ed20 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:38.632003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6244ae-cfa9-457a-976e-5a78e6c0ed20 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:38.634603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:38.672636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17795d0-5c2b-4ee5-90d9-4cfc1f2e1672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:38.673317 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17795d0-5c2b-4ee5-90d9-4cfc1f2e1672 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:38.673600 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17795d0-5c2b-4ee5-90d9-4cfc1f2e1672 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:38.673933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17795d0-5c2b-4ee5-90d9-4cfc1f2e1672 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:38.674771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17795d0-5c2b-4ee5-90d9-4cfc1f2e1672 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:38.675112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17795d0-5c2b-4ee5-90d9-4cfc1f2e1672 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:38.676775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17795d0-5c2b-4ee5-90d9-4cfc1f2e1672 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:38.690670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 411/836] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:38.696394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fe904-42c7-44f0-a0d3-ee1618dcd118 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:38.805424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1fe904-42c7-44f0-a0d3-ee1618dcd118 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:38.817699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:38.842742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de3871-9780-419e-8392-28e5a24691b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:38.843642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3871-9780-419e-8392-28e5a24691b5 None admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:38.843946 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3871-9780-419e-8392-28e5a24691b5 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:38.844234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de3871-9780-419e-8392-28e5a24691b5 None admin] PATH_INFO: `/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/0cd1747822c3493e8aec150a12409fbd/roles/2db2bc363aa74930932c823dc291d7f3` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:38.860182 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de3871-9780-419e-8392-28e5a24691b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=de62d2fb17f846f2bce5bef108353fd8, user_id=0cd1747822c3493e8aec150a12409fbd, role_id=2db2bc363aa74930932c823dc291d7f3)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:38.861662 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de3871-9780-419e-8392-28e5a24691b5 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:38.880044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 426/837] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:38:38 2022] PUT /identity/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/0cd1747822c3493e8aec150a12409fbd/roles/2db2bc363aa74930932c823dc291d7f3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:38:39.037705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76937b17-da7d-43db-b1d5-0ecc4ceef346 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:39.038175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76937b17-da7d-43db-b1d5-0ecc4ceef346 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:39.038419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76937b17-da7d-43db-b1d5-0ecc4ceef346 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:39.039984 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 412/838] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:39.048533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dbe6b-c52b-4d3c-98ab-9fda4191605f None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:39.049270 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dbe6b-c52b-4d3c-98ab-9fda4191605f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:39.049928 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2dbe6b-c52b-4d3c-98ab-9fda4191605f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:39.071743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-0b2dbe6b-c52b-4d3c-98ab-9fda4191605f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:39.073395 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0b2dbe6b-c52b-4d3c-98ab-9fda4191605f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:39.152047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 427/839] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:39.171027 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76937b17-da7d-43db-b1d5-0ecc4ceef346 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:39.276583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76937b17-da7d-43db-b1d5-0ecc4ceef346 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:39.312691 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fbb77-c0e0-42ba-9559-4ff05b7a62e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:39.313411 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fbb77-c0e0-42ba-9559-4ff05b7a62e1 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:39.313701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fbb77-c0e0-42ba-9559-4ff05b7a62e1 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:39.313980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fbb77-c0e0-42ba-9559-4ff05b7a62e1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:39.314635 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fbb77-c0e0-42ba-9559-4ff05b7a62e1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:39.316107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fbb77-c0e0-42ba-9559-4ff05b7a62e1 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-373fbb77-c0e0-42ba-9559-4ff05b7a62e1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:39.328696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:39.329708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:39.330762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 413/840] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:38:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:39.338069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dbe6b-c52b-4d3c-98ab-9fda4191605f None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:39.438465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:39.456280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2dbe6b-c52b-4d3c-98ab-9fda4191605f None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:39.505435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81feb7-947a-4314-96e5-98a1b4b10922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:39.506129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81feb7-947a-4314-96e5-98a1b4b10922 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:39.506416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81feb7-947a-4314-96e5-98a1b4b10922 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:39.506696 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81feb7-947a-4314-96e5-98a1b4b10922 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:39.507424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81feb7-947a-4314-96e5-98a1b4b10922 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:39.507735 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81feb7-947a-4314-96e5-98a1b4b10922 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:39.509181 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c81feb7-947a-4314-96e5-98a1b4b10922 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:39.515084 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 428/841] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:38:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:39.521107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fbb77-c0e0-42ba-9559-4ff05b7a62e1 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:39.554247 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:39.673863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fbb77-c0e0-42ba-9559-4ff05b7a62e1 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:39.725103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce65861-47db-4a3d-9077-46455523b895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:39.726203 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce65861-47db-4a3d-9077-46455523b895 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:39.726652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce65861-47db-4a3d-9077-46455523b895 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:39.727116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce65861-47db-4a3d-9077-46455523b895 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:39.728271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce65861-47db-4a3d-9077-46455523b895 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:39.728750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce65861-47db-4a3d-9077-46455523b895 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:39.730898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce65861-47db-4a3d-9077-46455523b895 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:39.744126 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 414/842] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:38:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:39.922759 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa604a6-d164-4718-b25f-a2f0b9c710e9 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:39.923106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa604a6-d164-4718-b25f-a2f0b9c710e9 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:39.923325 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa604a6-d164-4718-b25f-a2f0b9c710e9 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:39.925061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 429/843] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:38:39.932804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fcf82-a473-490e-aa3f-ad3659283c82 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:39.933096 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fcf82-a473-490e-aa3f-ad3659283c82 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:39.933375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4fcf82-a473-490e-aa3f-ad3659283c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:39.954319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-af4fcf82-a473-490e-aa3f-ad3659283c82 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:39.955535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-af4fcf82-a473-490e-aa3f-ad3659283c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:40.043214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 415/844] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:39 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:40.080209 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa604a6-d164-4718-b25f-a2f0b9c710e9 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:40.291101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa604a6-d164-4718-b25f-a2f0b9c710e9 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:40.353350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f7b95-09c7-46dd-b16c-4be386b27cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:40.355214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f7b95-09c7-46dd-b16c-4be386b27cb8 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:40.356033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f7b95-09c7-46dd-b16c-4be386b27cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:40.356795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f7b95-09c7-46dd-b16c-4be386b27cb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:40.376025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f7b95-09c7-46dd-b16c-4be386b27cb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-e23f7b95-09c7-46dd-b16c-4be386b27cb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:40.393140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:40.394909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:40.397236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:40.397236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:40.397236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:40.397236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:40.397236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:40.397236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:40.397236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:40.397236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:40.397236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:40.397236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:40.397236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:38:40.397236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:40.401640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 430/845] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:38:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:40.410746 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fcf82-a473-490e-aa3f-ad3659283c82 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:40.451818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:40.566201 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0d9b6-b468-490d-8324-03d57dead765 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:40.566948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0d9b6-b468-490d-8324-03d57dead765 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:40.566948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0d9b6-b468-490d-8324-03d57dead765 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:40.568312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 431/846] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:40.577373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd200846-70d2-448e-a6ae-180dab1ff655 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:40.577807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd200846-70d2-448e-a6ae-180dab1ff655 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:40.578198 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd200846-70d2-448e-a6ae-180dab1ff655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:40.593120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4fcf82-a473-490e-aa3f-ad3659283c82 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:40.596702 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-fd200846-70d2-448e-a6ae-180dab1ff655 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:40.598155 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-fd200846-70d2-448e-a6ae-180dab1ff655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:40.609241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:40.655583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fe7d2-5fce-4783-ab19-d3e2cd9caf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:40.656800 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fe7d2-5fce-4783-ab19-d3e2cd9caf36 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:40.657166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fe7d2-5fce-4783-ab19-d3e2cd9caf36 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:40.657521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fe7d2-5fce-4783-ab19-d3e2cd9caf36 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:40.658634 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fe7d2-5fce-4783-ab19-d3e2cd9caf36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:40.659024 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fe7d2-5fce-4783-ab19-d3e2cd9caf36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:40.660686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fe7d2-5fce-4783-ab19-d3e2cd9caf36 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:40.675668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 416/847] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:38:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:40.679165 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fe7d2-5fce-4783-ab19-d3e2cd9caf36 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:40.687865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 432/848] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:38:40.711467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd200846-70d2-448e-a6ae-180dab1ff655 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:40.801260 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:40.820479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fe7d2-5fce-4783-ab19-d3e2cd9caf36 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:40.844815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd200846-70d2-448e-a6ae-180dab1ff655 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:40.861777 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616e4d0-f561-4366-8021-19f48c1f7203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:40.862405 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616e4d0-f561-4366-8021-19f48c1f7203 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:40.862696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616e4d0-f561-4366-8021-19f48c1f7203 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:40.862980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616e4d0-f561-4366-8021-19f48c1f7203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:40.876650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616e4d0-f561-4366-8021-19f48c1f7203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:40.878072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616e4d0-f561-4366-8021-19f48c1f7203 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:40.886878 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a7c40-e452-4a67-a760-c520631cbf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:40.888052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a7c40-e452-4a67-a760-c520631cbf11 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:40.888640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a7c40-e452-4a67-a760-c520631cbf11 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:40.889123 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a7c40-e452-4a67-a760-c520631cbf11 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:40.890157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a7c40-e452-4a67-a760-c520631cbf11 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:40.891789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a7c40-e452-4a67-a760-c520631cbf11 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-2616e4d0-f561-4366-8021-19f48c1f7203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:40.896761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:40.897767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:40.898692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:40.898692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:40.898692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:40.898692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:40.900080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 417/849] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:40.904648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616e4d0-f561-4366-8021-19f48c1f7203 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:40.919200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 433/850] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Jul 14 17:38:40 2022] POST /identity/v3/services => generated 269 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:38:41.049411 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616e4d0-f561-4366-8021-19f48c1f7203 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:41.091234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:41.100997 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c17a3a5-5611-4f6b-bd45-2d9776240be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:41.101343 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17a3a5-5611-4f6b-bd45-2d9776240be6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:41.101590 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17a3a5-5611-4f6b-bd45-2d9776240be6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:41.101863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c17a3a5-5611-4f6b-bd45-2d9776240be6 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:41.102593 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17a3a5-5611-4f6b-bd45-2d9776240be6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:41.102901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17a3a5-5611-4f6b-bd45-2d9776240be6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:41.105403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c17a3a5-5611-4f6b-bd45-2d9776240be6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:41.122623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 418/851] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:41.129159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a7c40-e452-4a67-a760-c520631cbf11 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:41.133951 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:41.167485 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86209a8-d70f-4212-a218-43d3bba3061b None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:41.167800 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86209a8-d70f-4212-a218-43d3bba3061b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:41.168085 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86209a8-d70f-4212-a218-43d3bba3061b None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:41.170340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 419/852] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:38:41.177212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97891b-17d3-4e27-a053-1ee556554ca0 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:41.177789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97891b-17d3-4e27-a053-1ee556554ca0 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:41.178256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97891b-17d3-4e27-a053-1ee556554ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:41.198556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-0d97891b-17d3-4e27-a053-1ee556554ca0 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:41.199975 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0d97891b-17d3-4e27-a053-1ee556554ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:41.278001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 420/853] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:41 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:38:41.303903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97891b-17d3-4e27-a053-1ee556554ca0 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:41.394569 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a7c40-e452-4a67-a760-c520631cbf11 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:41.421013 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97891b-17d3-4e27-a053-1ee556554ca0 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:41.437058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e946a5-3d93-4e68-9e64-1319ca5bc015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:41.437932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e946a5-3d93-4e68-9e64-1319ca5bc015 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:41.438157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e946a5-3d93-4e68-9e64-1319ca5bc015 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:41.438439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e946a5-3d93-4e68-9e64-1319ca5bc015 None admin] PATH_INFO: `/v3/users/swift` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:41.446423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e946a5-3d93-4e68-9e64-1319ca5bc015 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:41.447875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e946a5-3d93-4e68-9e64-1319ca5bc015 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:41.449594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:41.464150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7861b-56e4-4077-a5fe-3fb720203060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:41.465415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7861b-56e4-4077-a5fe-3fb720203060 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-84e946a5-3d93-4e68-9e64-1319ca5bc015 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:41.466049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:41.468560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:41.470427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:41.470427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:41.470427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:41.470427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:41.470427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:41.470427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:41.470427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:41.470427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:41.470427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 17:38:41.470427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:41.470427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 434/854] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:38:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:41.472910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7861b-56e4-4077-a5fe-3fb720203060 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:41.472910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a7861b-56e4-4077-a5fe-3fb720203060 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:41.473546 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e946a5-3d93-4e68-9e64-1319ca5bc015 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:41.485732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a7861b-56e4-4077-a5fe-3fb720203060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-94a7861b-56e4-4077-a5fe-3fb720203060 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:41.494256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:41.495282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:41.496160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:41.496160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:41.496160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:41.496160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:41.496160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:41.496160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:41.496160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:41.496160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:41.496160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:41.496160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:41.496160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:38:41.496160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:41.504020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 421/855] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:38:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:41.505724 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7861b-56e4-4077-a5fe-3fb720203060 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:41.542508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:41.584502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e946a5-3d93-4e68-9e64-1319ca5bc015 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:41.618209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a7861b-56e4-4077-a5fe-3fb720203060 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:41.620631 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd02a1c-9ec3-402b-8224-3e20e41fc71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:41.621311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd02a1c-9ec3-402b-8224-3e20e41fc71c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:41.621599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd02a1c-9ec3-402b-8224-3e20e41fc71c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:41.622709 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd02a1c-9ec3-402b-8224-3e20e41fc71c None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:41.623565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd02a1c-9ec3-402b-8224-3e20e41fc71c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:41.623830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd02a1c-9ec3-402b-8224-3e20e41fc71c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:41.625323 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd02a1c-9ec3-402b-8224-3e20e41fc71c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:41.648375 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 435/856] 10.222.0.64 () {56 vars in 1135 bytes} [Thu Jul 14 17:38:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:41.650085 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd02a1c-9ec3-402b-8224-3e20e41fc71c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:41.659213 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccca80e-e3b6-4dcd-89de-f8804014eff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:41.660289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccca80e-e3b6-4dcd-89de-f8804014eff6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:41.660776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccca80e-e3b6-4dcd-89de-f8804014eff6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:41.661270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccca80e-e3b6-4dcd-89de-f8804014eff6 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:41.662322 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccca80e-e3b6-4dcd-89de-f8804014eff6 None admin] RBAC: Adding query filter params (name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:41.662870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccca80e-e3b6-4dcd-89de-f8804014eff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:41.664675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccca80e-e3b6-4dcd-89de-f8804014eff6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:41.675793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 422/857] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:38:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:41.681072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccca80e-e3b6-4dcd-89de-f8804014eff6 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:41.732452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:41.758134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd02a1c-9ec3-402b-8224-3e20e41fc71c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:41.783119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:41.795734 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccca80e-e3b6-4dcd-89de-f8804014eff6 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:41.797405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3315e6d3-6866-4b93-b039-2e85e8f6818e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:41.798092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315e6d3-6866-4b93-b039-2e85e8f6818e None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:41.798384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315e6d3-6866-4b93-b039-2e85e8f6818e None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:41.798715 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3315e6d3-6866-4b93-b039-2e85e8f6818e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:41.837824 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a96b6-548e-45e5-969e-e55a58ec7e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:41.838989 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a96b6-548e-45e5-969e-e55a58ec7e99 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:41.839448 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a96b6-548e-45e5-969e-e55a58ec7e99 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:41.839881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a96b6-548e-45e5-969e-e55a58ec7e99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:41.853119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a96b6-548e-45e5-969e-e55a58ec7e99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:41.854919 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a96b6-548e-45e5-969e-e55a58ec7e99 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-240a96b6-548e-45e5-969e-e55a58ec7e99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:41.874439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:41.875583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:41.876744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:41.876744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:41.876744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:41.876744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:41.876987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315e6d3-6866-4b93-b039-2e85e8f6818e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:41.876987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3315e6d3-6866-4b93-b039-2e85e8f6818e None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:41.883158 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 423/858] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:41.884400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a96b6-548e-45e5-969e-e55a58ec7e99 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:41.981601 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 436/859] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:41 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 17:38:41.983525 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40723fc-feeb-4687-adb6-48a22fa17bed None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:41.983815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40723fc-feeb-4687-adb6-48a22fa17bed None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:41.984117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40723fc-feeb-4687-adb6-48a22fa17bed None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:41.985709 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 437/860] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:41.989466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40723fc-feeb-4687-adb6-48a22fa17bed None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:41.991649 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:41.995574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a96b6-548e-45e5-969e-e55a58ec7e99 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:42.015847 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:42.034132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6982b9-69d8-4e16-8bbd-7b3a5ee018eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:42.034848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6982b9-69d8-4e16-8bbd-7b3a5ee018eb None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:42.035147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6982b9-69d8-4e16-8bbd-7b3a5ee018eb None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:42.035428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6982b9-69d8-4e16-8bbd-7b3a5ee018eb None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:42.036187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6982b9-69d8-4e16-8bbd-7b3a5ee018eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:42.036500 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6982b9-69d8-4e16-8bbd-7b3a5ee018eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:42.037880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6982b9-69d8-4e16-8bbd-7b3a5ee018eb None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:42.057928 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 424/861] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:42.057928 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81262a7-ef8a-4d41-87f2-1316ad375979 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:42.057928 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81262a7-ef8a-4d41-87f2-1316ad375979 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:42.057928 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81262a7-ef8a-4d41-87f2-1316ad375979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:42.077877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-d81262a7-ef8a-4d41-87f2-1316ad375979 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:42.079119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d81262a7-ef8a-4d41-87f2-1316ad375979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:42.129586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40723fc-feeb-4687-adb6-48a22fa17bed None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:42.166111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 425/862] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:42 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:42.167169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81262a7-ef8a-4d41-87f2-1316ad375979 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:42.178955 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ae91f-76a3-46ad-a2bd-be8a68553f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:42.180182 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ae91f-76a3-46ad-a2bd-be8a68553f71 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:42.181050 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ae91f-76a3-46ad-a2bd-be8a68553f71 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:42.181517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ae91f-76a3-46ad-a2bd-be8a68553f71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:42.195795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ae91f-76a3-46ad-a2bd-be8a68553f71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:42.197302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ae91f-76a3-46ad-a2bd-be8a68553f71 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-be6ae91f-76a3-46ad-a2bd-be8a68553f71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:42.218415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:42.219533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:42.220490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:42.220490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:42.220490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:42.220490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:42.220490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 438/863] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:42.220490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ae91f-76a3-46ad-a2bd-be8a68553f71 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:42.268866 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:42.286077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81262a7-ef8a-4d41-87f2-1316ad375979 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:42.304884 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:42.321012 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde27317-8319-42be-945f-92350436be33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:42.321836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde27317-8319-42be-945f-92350436be33 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:42.322175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde27317-8319-42be-945f-92350436be33 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:42.322415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde27317-8319-42be-945f-92350436be33 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:42.331273 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde27317-8319-42be-945f-92350436be33 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:42.332759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde27317-8319-42be-945f-92350436be33 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:42.335186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ae91f-76a3-46ad-a2bd-be8a68553f71 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-dde27317-8319-42be-945f-92350436be33 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:42.348793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:42.349828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:42.350906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:42.350906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:42.350906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:42.350906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:42.350906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:42.350906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:42.350906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:42.350906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:42.350906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 17:38:42.350906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:42.350906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 426/864] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Jul 14 17:38:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:42.350906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde27317-8319-42be-945f-92350436be33 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:42.376795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccc994-7867-480a-99c3-34b775255ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:42.377460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccc994-7867-480a-99c3-34b775255ab7 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:42.377738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccc994-7867-480a-99c3-34b775255ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:42.378012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdccc994-7867-480a-99c3-34b775255ab7 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:42.378732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccc994-7867-480a-99c3-34b775255ab7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:42.380117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdccc994-7867-480a-99c3-34b775255ab7 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:42.397249 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 439/865] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Jul 14 17:38:42 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:38:42.397995 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccc994-7867-480a-99c3-34b775255ab7 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:42.498068 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde27317-8319-42be-945f-92350436be33 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:42.548549 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdccc994-7867-480a-99c3-34b775255ab7 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:42.549534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3f99b-fe84-4a84-ae57-50ca98ec88b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:42.550303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3f99b-fe84-4a84-ae57-50ca98ec88b1 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:42.550672 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3f99b-fe84-4a84-ae57-50ca98ec88b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:42.551023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3f99b-fe84-4a84-ae57-50ca98ec88b1 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:42.553029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3f99b-fe84-4a84-ae57-50ca98ec88b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:42.553029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3f99b-fe84-4a84-ae57-50ca98ec88b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:42.555890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3f99b-fe84-4a84-ae57-50ca98ec88b1 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:42.573953 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 427/866] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:42.580362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3f99b-fe84-4a84-ae57-50ca98ec88b1 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:42.590650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:42.596748 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:42.605633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38462386-104b-4b9a-a521-530a50187580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:42.606755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38462386-104b-4b9a-a521-530a50187580 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:42.607546 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38462386-104b-4b9a-a521-530a50187580 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:42.607822 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38462386-104b-4b9a-a521-530a50187580 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:42.608741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38462386-104b-4b9a-a521-530a50187580 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:42.609055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38462386-104b-4b9a-a521-530a50187580 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:42.610492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38462386-104b-4b9a-a521-530a50187580 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:42.633446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 440/867] 10.222.0.64 () {56 vars in 1149 bytes} [Thu Jul 14 17:38:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:42.638554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38462386-104b-4b9a-a521-530a50187580 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:42.727080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3f99b-fe84-4a84-ae57-50ca98ec88b1 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:42.761547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38462386-104b-4b9a-a521-530a50187580 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:42.769058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab6921-f345-4fe5-82da-5db106e80405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:42.770763 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab6921-f345-4fe5-82da-5db106e80405 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:42.771286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab6921-f345-4fe5-82da-5db106e80405 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:42.771769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab6921-f345-4fe5-82da-5db106e80405 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:42.782977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab6921-f345-4fe5-82da-5db106e80405 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:42.785476 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab6921-f345-4fe5-82da-5db106e80405 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:42.800166 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86775fb-ec89-4ecc-a018-5eeab0c0a26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:42.800854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86775fb-ec89-4ecc-a018-5eeab0c0a26f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:42.801172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86775fb-ec89-4ecc-a018-5eeab0c0a26f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:42.801449 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86775fb-ec89-4ecc-a018-5eeab0c0a26f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-b2ab6921-f345-4fe5-82da-5db106e80405 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:42.807141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:42.808102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:42.809019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:42.809019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:42.809019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:42.809019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:42.809019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:42.809019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:42.809019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:42.809019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:42.812785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 428/868] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:42.818735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab6921-f345-4fe5-82da-5db106e80405 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:42.868439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:42.884539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86775fb-ec89-4ecc-a018-5eeab0c0a26f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:42.886577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86775fb-ec89-4ecc-a018-5eeab0c0a26f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:42.901697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:42.943212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab6921-f345-4fe5-82da-5db106e80405 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:42.990738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56ae2b-31dd-4dea-a7ea-667c19852ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:42.991412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56ae2b-31dd-4dea-a7ea-667c19852ec8 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:42.991695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56ae2b-31dd-4dea-a7ea-667c19852ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:42.991999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56ae2b-31dd-4dea-a7ea-667c19852ec8 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:42.992797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56ae2b-31dd-4dea-a7ea-667c19852ec8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:42.993109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56ae2b-31dd-4dea-a7ea-667c19852ec8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:42.994596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56ae2b-31dd-4dea-a7ea-667c19852ec8 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:43.008353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 441/869] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:42 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 17:38:43.009396 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 429/870] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:43.010219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86775fb-ec89-4ecc-a018-5eeab0c0a26f None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:43.018941 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56ae2b-31dd-4dea-a7ea-667c19852ec8 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:43.135723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:43.151788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86775fb-ec89-4ecc-a018-5eeab0c0a26f None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:43.154656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56ae2b-31dd-4dea-a7ea-667c19852ec8 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:43.184943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:43.195255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d725c96a-133f-44e6-bcba-ae355a52176c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:43.195414 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae02a8-447a-4243-bf1b-dd0755811f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:43.195943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725c96a-133f-44e6-bcba-ae355a52176c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:43.196071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae02a8-447a-4243-bf1b-dd0755811f83 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:43.196229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725c96a-133f-44e6-bcba-ae355a52176c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:43.196393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae02a8-447a-4243-bf1b-dd0755811f83 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:43.196577 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d725c96a-133f-44e6-bcba-ae355a52176c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:43.196673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae02a8-447a-4243-bf1b-dd0755811f83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:43.197332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725c96a-133f-44e6-bcba-ae355a52176c None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=97f15306528e415baa5c69f12f22c75e) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:43.197650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725c96a-133f-44e6-bcba-ae355a52176c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:43.199073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d725c96a-133f-44e6-bcba-ae355a52176c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:43.212118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 430/871] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:43 2022] GET /identity/v3/role_assignments?user.id=97f15306528e415baa5c69f12f22c75e&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:43.212313 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae02a8-447a-4243-bf1b-dd0755811f83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:43.212417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9eca1-04de-4a4d-b609-e23889952575 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:43.212417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9eca1-04de-4a4d-b609-e23889952575 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:43.212417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f9eca1-04de-4a4d-b609-e23889952575 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:43.212598 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae02a8-447a-4243-bf1b-dd0755811f83 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:43.212666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 431/872] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:43.214830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd988ea-c407-4b96-bce8-669ff3c051d3 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:43.215621 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd988ea-c407-4b96-bce8-669ff3c051d3 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:43.215621 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd988ea-c407-4b96-bce8-669ff3c051d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-edae02a8-447a-4243-bf1b-dd0755811f83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:43.233000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:43.233966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:43.235417 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:43.235417 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:43.235417 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:43.235417 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:43.235720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-bfd988ea-c407-4b96-bce8-669ff3c051d3 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:43.235816 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 442/873] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:43.235897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-bfd988ea-c407-4b96-bce8-669ff3c051d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:43.239808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae02a8-447a-4243-bf1b-dd0755811f83 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:43.328360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 432/874] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:43.349023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd988ea-c407-4b96-bce8-669ff3c051d3 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:43.370462 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae02a8-447a-4243-bf1b-dd0755811f83 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:43.422467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83fa8fb-ee85-4578-a4d7-2f8ed6224069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:43.423149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fa8fb-ee85-4578-a4d7-2f8ed6224069 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:43.423435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fa8fb-ee85-4578-a4d7-2f8ed6224069 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:43.423716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83fa8fb-ee85-4578-a4d7-2f8ed6224069 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:43.424509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fa8fb-ee85-4578-a4d7-2f8ed6224069 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:43.424820 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fa8fb-ee85-4578-a4d7-2f8ed6224069 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:43.426207 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83fa8fb-ee85-4578-a4d7-2f8ed6224069 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:43.437682 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:43.443023 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 443/875] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:43.448497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83fa8fb-ee85-4578-a4d7-2f8ed6224069 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:43.484588 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd988ea-c407-4b96-bce8-669ff3c051d3 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:43.512897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:43.535891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63edc307-c145-4102-86e0-bb00d08ec888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:43.536773 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edc307-c145-4102-86e0-bb00d08ec888 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:43.537054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edc307-c145-4102-86e0-bb00d08ec888 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:43.537330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edc307-c145-4102-86e0-bb00d08ec888 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:43.538002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63edc307-c145-4102-86e0-bb00d08ec888 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:43.539763 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63edc307-c145-4102-86e0-bb00d08ec888 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-63edc307-c145-4102-86e0-bb00d08ec888 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:43.557695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:43.558782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:43.560061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 433/876] 10.222.0.64 () {56 vars in 1132 bytes} [Thu Jul 14 17:38:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:43.564989 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63edc307-c145-4102-86e0-bb00d08ec888 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:43.581736 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83fa8fb-ee85-4578-a4d7-2f8ed6224069 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:43.626252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56179d-9fcc-46d5-802c-bc0b5b634c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:43.627004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56179d-9fcc-46d5-802c-bc0b5b634c56 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:43.627556 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56179d-9fcc-46d5-802c-bc0b5b634c56 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:43.627843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc56179d-9fcc-46d5-802c-bc0b5b634c56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:43.637148 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56179d-9fcc-46d5-802c-bc0b5b634c56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:43.639295 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc56179d-9fcc-46d5-802c-bc0b5b634c56 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-fc56179d-9fcc-46d5-802c-bc0b5b634c56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:43.660846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:43.661809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:43.663043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:43.663043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:43.663043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:43.663043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:43.663043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:43.663043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:43.663043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:43.663043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:43.663043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 444/877] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:43.673046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56179d-9fcc-46d5-802c-bc0b5b634c56 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:43.700080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63edc307-c145-4102-86e0-bb00d08ec888 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:43.734491 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:43.740925 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d19d18-7c5b-407c-b6f0-761ec364baf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:43.741654 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d19d18-7c5b-407c-b6f0-761ec364baf6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:43.741941 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d19d18-7c5b-407c-b6f0-761ec364baf6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:43.742219 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d19d18-7c5b-407c-b6f0-761ec364baf6 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:43.743150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d19d18-7c5b-407c-b6f0-761ec364baf6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:43.743466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d19d18-7c5b-407c-b6f0-761ec364baf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:43.744997 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d19d18-7c5b-407c-b6f0-761ec364baf6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:43.751376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 434/878] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Jul 14 17:38:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:43.756749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d19d18-7c5b-407c-b6f0-761ec364baf6 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:43.780048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc56179d-9fcc-46d5-802c-bc0b5b634c56 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:43.817419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:43.827563 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2e459-97d4-4405-9948-5814db4d6624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:43.828703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2e459-97d4-4405-9948-5814db4d6624 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:43.828993 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2e459-97d4-4405-9948-5814db4d6624 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:43.829279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2e459-97d4-4405-9948-5814db4d6624 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:43.830368 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2e459-97d4-4405-9948-5814db4d6624 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:43.830684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2e459-97d4-4405-9948-5814db4d6624 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:43.832839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2e459-97d4-4405-9948-5814db4d6624 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:43.852449 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 445/879] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:43.858395 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2e459-97d4-4405-9948-5814db4d6624 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:43.880890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d19d18-7c5b-407c-b6f0-761ec364baf6 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:43.915656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920215c-a0e6-46d6-a569-fb23ef788b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:43.916765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920215c-a0e6-46d6-a569-fb23ef788b54 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:43.917247 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920215c-a0e6-46d6-a569-fb23ef788b54 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:43.917739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920215c-a0e6-46d6-a569-fb23ef788b54 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:43.918660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920215c-a0e6-46d6-a569-fb23ef788b54 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:43.919163 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920215c-a0e6-46d6-a569-fb23ef788b54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:43.921333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920215c-a0e6-46d6-a569-fb23ef788b54 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:43.927299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 435/880] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Jul 14 17:38:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:43.932628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920215c-a0e6-46d6-a569-fb23ef788b54 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:43.989660 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2e459-97d4-4405-9948-5814db4d6624 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:44.015607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:44.031462 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfda06-c825-4dc5-92b8-bb42264aef8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:44.032539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfda06-c825-4dc5-92b8-bb42264aef8d None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:44.033066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfda06-c825-4dc5-92b8-bb42264aef8d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:44.033563 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfda06-c825-4dc5-92b8-bb42264aef8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:44.034611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfda06-c825-4dc5-92b8-bb42264aef8d None admin] RBAC: Adding query filter params (role.id=2db2bc363aa74930932c823dc291d7f3, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=0cd1747822c3493e8aec150a12409fbd) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:44.035135 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfda06-c825-4dc5-92b8-bb42264aef8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:44.036821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfda06-c825-4dc5-92b8-bb42264aef8d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:44.043876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 446/881] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:43 2022] GET /identity/v3/role_assignments?user.id=0cd1747822c3493e8aec150a12409fbd&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=2db2bc363aa74930932c823dc291d7f3 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:44.073718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920215c-a0e6-46d6-a569-fb23ef788b54 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:44.089170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:44.113884 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a4979-30cc-4e3e-a024-5b2644825700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:44.114606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a4979-30cc-4e3e-a024-5b2644825700 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:44.114933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a4979-30cc-4e3e-a024-5b2644825700 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:44.115211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a4979-30cc-4e3e-a024-5b2644825700 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:44.116040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a4979-30cc-4e3e-a024-5b2644825700 None admin] RBAC: Adding query filter params (interface=public, service_id=79a2dddd69bf4cd5b64be34f64d1b4ac, region_id=RegionOne) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:44.116355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a4979-30cc-4e3e-a024-5b2644825700 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:44.117792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a4979-30cc-4e3e-a024-5b2644825700 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:44.126074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 436/882] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Jul 14 17:38:43 2022] GET /identity/v3/endpoints?service_id=79a2dddd69bf4cd5b64be34f64d1b4ac&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:44.133238 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfda06-c825-4dc5-92b8-bb42264aef8d None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:44.250929 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfda06-c825-4dc5-92b8-bb42264aef8d None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:44.287757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdd3a4f-eff9-4076-b9f9-8cc90bfa2bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:44.288551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd3a4f-eff9-4076-b9f9-8cc90bfa2bbd None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:44.288836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd3a4f-eff9-4076-b9f9-8cc90bfa2bbd None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:44.289113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdd3a4f-eff9-4076-b9f9-8cc90bfa2bbd None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:44.289810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd3a4f-eff9-4076-b9f9-8cc90bfa2bbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:44.291186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdd3a4f-eff9-4076-b9f9-8cc90bfa2bbd None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:44.297754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 447/883] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Jul 14 17:38:44 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:38:44.649688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf6a79-5a42-470d-b727-c136261619f3 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:44.650208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf6a79-5a42-470d-b727-c136261619f3 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:44.650287 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf6a79-5a42-470d-b727-c136261619f3 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:44.651770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 437/884] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:44.657099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a31a7-b98e-4a0a-ab78-ef1ce7ca0ea9 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:44.657540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a31a7-b98e-4a0a-ab78-ef1ce7ca0ea9 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:44.657632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428a31a7-b98e-4a0a-ab78-ef1ce7ca0ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:44.675439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-428a31a7-b98e-4a0a-ab78-ef1ce7ca0ea9 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:44.676750 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-428a31a7-b98e-4a0a-ab78-ef1ce7ca0ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:44.723003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:44.749468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 448/885] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:44.772560 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf6a79-5a42-470d-b727-c136261619f3 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:44.883681 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf6a79-5a42-470d-b727-c136261619f3 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:44.919114 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfba609-2240-4bcd-beb8-1c4f7835af3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:44.920468 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfba609-2240-4bcd-beb8-1c4f7835af3b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:44.920755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfba609-2240-4bcd-beb8-1c4f7835af3b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:44.921031 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfba609-2240-4bcd-beb8-1c4f7835af3b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:44.921689 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfba609-2240-4bcd-beb8-1c4f7835af3b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:44.923188 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfba609-2240-4bcd-beb8-1c4f7835af3b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-7cfba609-2240-4bcd-beb8-1c4f7835af3b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:44.936871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:44.937921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:44.938761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 438/886] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:38:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:44.944575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a31a7-b98e-4a0a-ab78-ef1ce7ca0ea9 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:45.056219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428a31a7-b98e-4a0a-ab78-ef1ce7ca0ea9 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:45.100955 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01ccc7-1d37-4484-a4df-b1ab1c771898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:45.101644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01ccc7-1d37-4484-a4df-b1ab1c771898 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:45.101931 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01ccc7-1d37-4484-a4df-b1ab1c771898 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:45.102208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b01ccc7-1d37-4484-a4df-b1ab1c771898 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:45.102939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01ccc7-1d37-4484-a4df-b1ab1c771898 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:45.103248 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01ccc7-1d37-4484-a4df-b1ab1c771898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:45.104664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b01ccc7-1d37-4484-a4df-b1ab1c771898 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:45.110231 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 449/887] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:38:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:45.115757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfba609-2240-4bcd-beb8-1c4f7835af3b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:45.167484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:45.230974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfba609-2240-4bcd-beb8-1c4f7835af3b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:45.267508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f06095-14ef-4415-a12a-6c14235539d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:45.268492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f06095-14ef-4415-a12a-6c14235539d7 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:45.268940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f06095-14ef-4415-a12a-6c14235539d7 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:45.269371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f06095-14ef-4415-a12a-6c14235539d7 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:45.270282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f06095-14ef-4415-a12a-6c14235539d7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:45.270760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f06095-14ef-4415-a12a-6c14235539d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:45.272320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f06095-14ef-4415-a12a-6c14235539d7 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:45.277983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 439/888] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:38:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:45.284230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01ccc7-1d37-4484-a4df-b1ab1c771898 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:45.342590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:45.398339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b01ccc7-1d37-4484-a4df-b1ab1c771898 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:45.399670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454357c6-4cfd-43b0-a5fe-bcec35010b1c None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:45.400105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454357c6-4cfd-43b0-a5fe-bcec35010b1c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:45.400291 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454357c6-4cfd-43b0-a5fe-bcec35010b1c None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:45.401922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 440/889] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:45.409457 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27749f-39e8-4987-984c-e46c35bf06cb None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:45.410111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27749f-39e8-4987-984c-e46c35bf06cb None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:45.410604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27749f-39e8-4987-984c-e46c35bf06cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:45.429576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-7e27749f-39e8-4987-984c-e46c35bf06cb None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:45.431307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7e27749f-39e8-4987-984c-e46c35bf06cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:45.436609 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b5f30-5525-41ab-a5cc-0713139be31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:45.436945 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b5f30-5525-41ab-a5cc-0713139be31c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:45.437224 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b5f30-5525-41ab-a5cc-0713139be31c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:45.437532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b5f30-5525-41ab-a5cc-0713139be31c None admin] PATH_INFO: `/v3/endpoints` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:45.439895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b5f30-5525-41ab-a5cc-0713139be31c None admin] RBAC: Adding query filter params (interface=public, service_id=53bb8b6acead4939b0521a21896ad202, region_id=RegionOne) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:45.439895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b5f30-5525-41ab-a5cc-0713139be31c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:45.440092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b5f30-5525-41ab-a5cc-0713139be31c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:45.445893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 450/890] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Jul 14 17:38:45 2022] GET /identity/v3/endpoints?service_id=53bb8b6acead4939b0521a21896ad202&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:45.453159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b5f30-5525-41ab-a5cc-0713139be31c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:45.532426 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 441/891] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:45.556789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27749f-39e8-4987-984c-e46c35bf06cb None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:45.575717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b5f30-5525-41ab-a5cc-0713139be31c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:45.598256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:45.612301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:45.614474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b1a15-1b71-4e24-9d00-34e06cc26978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:45.615159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b1a15-1b71-4e24-9d00-34e06cc26978 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:45.615435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b1a15-1b71-4e24-9d00-34e06cc26978 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:45.615704 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b1a15-1b71-4e24-9d00-34e06cc26978 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:45.616428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b1a15-1b71-4e24-9d00-34e06cc26978 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:45.617789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b1a15-1b71-4e24-9d00-34e06cc26978 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:45.625033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 451/892] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Jul 14 17:38:45 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:38:45.681660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27749f-39e8-4987-984c-e46c35bf06cb None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:45.724534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87187d02-407a-4b42-ae99-0c3143efe7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:45.725747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87187d02-407a-4b42-ae99-0c3143efe7c8 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:45.726246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87187d02-407a-4b42-ae99-0c3143efe7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:45.726723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87187d02-407a-4b42-ae99-0c3143efe7c8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:45.727613 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87187d02-407a-4b42-ae99-0c3143efe7c8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:45.729251 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87187d02-407a-4b42-ae99-0c3143efe7c8 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-87187d02-407a-4b42-ae99-0c3143efe7c8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:45.743127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:45.744332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:45.745264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:45.745264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:45.745264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:45.745264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:45.745264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:45.745264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:45.745264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:45.745264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:45.745264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:45.745264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:45.745264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:45.745264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:45.745264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:38:45.745264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:45.748652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 442/893] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Jul 14 17:38:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:45.758620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b1a15-1b71-4e24-9d00-34e06cc26978 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:45.894638 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b1a15-1b71-4e24-9d00-34e06cc26978 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:45.938274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc8cd78-b56e-4c06-8e98-2ebd362cd82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:45.938998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8cd78-b56e-4c06-8e98-2ebd362cd82f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:45.939289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8cd78-b56e-4c06-8e98-2ebd362cd82f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:45.939571 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc8cd78-b56e-4c06-8e98-2ebd362cd82f None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:45.940394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8cd78-b56e-4c06-8e98-2ebd362cd82f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:45.940707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8cd78-b56e-4c06-8e98-2ebd362cd82f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:45.942209 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc8cd78-b56e-4c06-8e98-2ebd362cd82f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:45.948433 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 452/894] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Jul 14 17:38:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:45.954906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87187d02-407a-4b42-ae99-0c3143efe7c8 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:46.067116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:46.071182 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87187d02-407a-4b42-ae99-0c3143efe7c8 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:46.111674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20230194-09d5-4277-90d1-90b341b2bd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:46.112390 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20230194-09d5-4277-90d1-90b341b2bd2a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:46.112672 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20230194-09d5-4277-90d1-90b341b2bd2a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:46.112952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20230194-09d5-4277-90d1-90b341b2bd2a None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:46.113673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20230194-09d5-4277-90d1-90b341b2bd2a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:46.113984 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20230194-09d5-4277-90d1-90b341b2bd2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:46.115612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20230194-09d5-4277-90d1-90b341b2bd2a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:46.121606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 443/895] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Jul 14 17:38:45 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:46.361352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995b335-a287-4009-bf81-d3cd29a5e84e None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:46.361709 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995b335-a287-4009-bf81-d3cd29a5e84e None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:46.361909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c995b335-a287-4009-bf81-d3cd29a5e84e None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:46.363632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 453/896] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:46.369037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb8c0c-b9eb-47d4-abf3-b66a4c5f7ea4 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:46.369330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb8c0c-b9eb-47d4-abf3-b66a4c5f7ea4 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:46.369617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb8c0c-b9eb-47d4-abf3-b66a4c5f7ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:46.387454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-ceeb8c0c-b9eb-47d4-abf3-b66a4c5f7ea4 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:46.389242 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ceeb8c0c-b9eb-47d4-abf3-b66a4c5f7ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:46.462795 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 444/897] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:46.487146 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995b335-a287-4009-bf81-d3cd29a5e84e None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:46.524915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:46.605106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c995b335-a287-4009-bf81-d3cd29a5e84e None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:46.641488 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7d11be-718f-405d-92be-c40913741946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:46.642215 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d11be-718f-405d-92be-c40913741946 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:46.642490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d11be-718f-405d-92be-c40913741946 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:46.642762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7d11be-718f-405d-92be-c40913741946 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:46.653531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7d11be-718f-405d-92be-c40913741946 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-9b7d11be-718f-405d-92be-c40913741946 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:46.662235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:46.663410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:46.664632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:46.664632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:46.664632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:46.664632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:46.664632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:46.664632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:46.664632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:46.664632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:46.664632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:46.664632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:46.664632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 17:38:46.664632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:46.664632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 454/898] 10.222.0.64 () {56 vars in 1120 bytes} [Thu Jul 14 17:38:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:46.673218 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb8c0c-b9eb-47d4-abf3-b66a4c5f7ea4 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:46.684337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62696f-2228-4301-b23b-49f466759e1f None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:46.684643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62696f-2228-4301-b23b-49f466759e1f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:46.684888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62696f-2228-4301-b23b-49f466759e1f None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:46.686930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 455/899] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:38:46.691982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4f6b4-cf97-4acd-bb46-367783be3153 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:46.692272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4f6b4-cf97-4acd-bb46-367783be3153 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:46.692556 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d4f6b4-cf97-4acd-bb46-367783be3153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:46.710013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-a3d4f6b4-cf97-4acd-bb46-367783be3153 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:46.711237 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a3d4f6b4-cf97-4acd-bb46-367783be3153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:46.778262 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:46.789885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb8c0c-b9eb-47d4-abf3-b66a4c5f7ea4 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:46.791432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 456/900] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:38:46.814811 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4f6b4-cf97-4acd-bb46-367783be3153 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:46.831573 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b4cfc-0b68-4fb9-b770-0f04c1e0408c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:46.832699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4cfc-0b68-4fb9-b770-0f04c1e0408c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:46.833259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4cfc-0b68-4fb9-b770-0f04c1e0408c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:46.833864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4cfc-0b68-4fb9-b770-0f04c1e0408c None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:46.834830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b4cfc-0b68-4fb9-b770-0f04c1e0408c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:46.835369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b4cfc-0b68-4fb9-b770-0f04c1e0408c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:46.837064 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b4cfc-0b68-4fb9-b770-0f04c1e0408c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:46.841817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:46.847425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 445/901] 10.222.0.64 () {56 vars in 1115 bytes} [Thu Jul 14 17:38:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:46.853845 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b4cfc-0b68-4fb9-b770-0f04c1e0408c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:46.942430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d4f6b4-cf97-4acd-bb46-367783be3153 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:46.979471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c107845-0c9b-4110-9db4-ad9df26691c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:46.980480 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c107845-0c9b-4110-9db4-ad9df26691c6 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:46.980624 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c107845-0c9b-4110-9db4-ad9df26691c6 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:46.980863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c107845-0c9b-4110-9db4-ad9df26691c6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:46.981531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c107845-0c9b-4110-9db4-ad9df26691c6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:46.982942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c107845-0c9b-4110-9db4-ad9df26691c6 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:46.984303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b4cfc-0b68-4fb9-b770-0f04c1e0408c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-9c107845-0c9b-4110-9db4-ad9df26691c6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:46.995707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:46.996719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:46.998502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 457/902] 10.222.0.64 () {56 vars in 1132 bytes} [Thu Jul 14 17:38:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:47.008889 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c107845-0c9b-4110-9db4-ad9df26691c6 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:47.028862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030a86c-56bd-4936-bc37-cca8f3bd7e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:47.031400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030a86c-56bd-4936-bc37-cca8f3bd7e2b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:47.032249 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030a86c-56bd-4936-bc37-cca8f3bd7e2b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:47.032759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2030a86c-56bd-4936-bc37-cca8f3bd7e2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:47.048070 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030a86c-56bd-4936-bc37-cca8f3bd7e2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:47.050451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2030a86c-56bd-4936-bc37-cca8f3bd7e2b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-2030a86c-56bd-4936-bc37-cca8f3bd7e2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:47.073083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:47.074022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:47.074920 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:47.074920 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:47.074920 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:47.074920 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:47.078103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 446/903] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:47.083968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030a86c-56bd-4936-bc37-cca8f3bd7e2b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:47.098382 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:47.147405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c107845-0c9b-4110-9db4-ad9df26691c6 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:47.160296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:47.186109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a4db6-1371-4306-9e5a-f432c4d6eb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:47.187291 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a4db6-1371-4306-9e5a-f432c4d6eb07 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:47.187898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a4db6-1371-4306-9e5a-f432c4d6eb07 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:47.188414 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a4db6-1371-4306-9e5a-f432c4d6eb07 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:47.189356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a4db6-1371-4306-9e5a-f432c4d6eb07 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:47.189879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a4db6-1371-4306-9e5a-f432c4d6eb07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:47.191854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a4db6-1371-4306-9e5a-f432c4d6eb07 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:47.194989 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2030a86c-56bd-4936-bc37-cca8f3bd7e2b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:47.198273 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 458/904] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Jul 14 17:38:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:47.205062 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a4db6-1371-4306-9e5a-f432c4d6eb07 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:47.246573 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d967d-4b8b-4d33-884e-5a46e0105fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:47.248807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d967d-4b8b-4d33-884e-5a46e0105fcc None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:47.249315 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d967d-4b8b-4d33-884e-5a46e0105fcc None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:47.249795 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d967d-4b8b-4d33-884e-5a46e0105fcc None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:47.251950 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d967d-4b8b-4d33-884e-5a46e0105fcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:47.252519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d967d-4b8b-4d33-884e-5a46e0105fcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:47.254164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85d967d-4b8b-4d33-884e-5a46e0105fcc None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:47.276751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 447/905] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:47.278096 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d967d-4b8b-4d33-884e-5a46e0105fcc None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:47.332677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a4db6-1371-4306-9e5a-f432c4d6eb07 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:47.368404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:47.379334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5fdc4-a371-4626-9567-4e5aa57c35c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:47.380041 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5fdc4-a371-4626-9567-4e5aa57c35c0 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:47.380326 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5fdc4-a371-4626-9567-4e5aa57c35c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:47.380612 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae5fdc4-a371-4626-9567-4e5aa57c35c0 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:47.381321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5fdc4-a371-4626-9567-4e5aa57c35c0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:47.381626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5fdc4-a371-4626-9567-4e5aa57c35c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:47.383021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae5fdc4-a371-4626-9567-4e5aa57c35c0 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:47.394172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 459/906] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Jul 14 17:38:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:47.395067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5fdc4-a371-4626-9567-4e5aa57c35c0 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:47.408770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d967d-4b8b-4d33-884e-5a46e0105fcc None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:47.416516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:47.457121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8b568-53d8-4394-acd0-8f5166b4bdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:47.458837 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8b568-53d8-4394-acd0-8f5166b4bdee None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:47.459594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8b568-53d8-4394-acd0-8f5166b4bdee None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:47.460168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d8b568-53d8-4394-acd0-8f5166b4bdee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:47.471561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8b568-53d8-4394-acd0-8f5166b4bdee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:47.474475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d8b568-53d8-4394-acd0-8f5166b4bdee None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-62d8b568-53d8-4394-acd0-8f5166b4bdee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:47.495176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:47.496235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:47.497195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:47.497195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:47.497195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:47.497195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:47.497195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:47.497195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:47.497195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:47.497195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:47.497195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 17:38:47.497195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:47.497757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 448/907] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Jul 14 17:38:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:47.505521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8b568-53d8-4394-acd0-8f5166b4bdee None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:47.520142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae5fdc4-a371-4626-9567-4e5aa57c35c0 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:47.565038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f165d-e2b8-44fa-a170-e1571216dddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:47.566332 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f165d-e2b8-44fa-a170-e1571216dddc None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:47.566979 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f165d-e2b8-44fa-a170-e1571216dddc None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:47.567765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f165d-e2b8-44fa-a170-e1571216dddc None admin] PATH_INFO: `/v3/endpoints` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:47.569044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f165d-e2b8-44fa-a170-e1571216dddc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:47.571105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f165d-e2b8-44fa-a170-e1571216dddc None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:47.596836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 460/908] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Jul 14 17:38:47 2022] POST /identity/v3/endpoints => generated 347 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:38:47.619764 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d8b568-53d8-4394-acd0-8f5166b4bdee None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:47.655093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:47.672922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c8a636-4d50-4700-9220-25cf3501c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:47.674041 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8a636-4d50-4700-9220-25cf3501c7f3 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:47.674532 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8a636-4d50-4700-9220-25cf3501c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:47.675011 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8a636-4d50-4700-9220-25cf3501c7f3 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:47.676477 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8a636-4d50-4700-9220-25cf3501c7f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:47.677063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8a636-4d50-4700-9220-25cf3501c7f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:47.679018 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c8a636-4d50-4700-9220-25cf3501c7f3 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:47.700650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 449/909] 10.222.0.64 () {56 vars in 1149 bytes} [Thu Jul 14 17:38:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:47.707457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f165d-e2b8-44fa-a170-e1571216dddc None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:47.788600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:47.821999 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f165d-e2b8-44fa-a170-e1571216dddc None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:47.856825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30958f-85f3-4253-b3b7-be3d5db86633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:47.858011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30958f-85f3-4253-b3b7-be3d5db86633 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:47.858327 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30958f-85f3-4253-b3b7-be3d5db86633 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:47.858616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30958f-85f3-4253-b3b7-be3d5db86633 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:47.869156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e1c84-6025-492b-acb1-7b8add0844b3 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:47.869696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e1c84-6025-492b-acb1-7b8add0844b3 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:47.869696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e1c84-6025-492b-acb1-7b8add0844b3 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:47.871340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 450/910] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:47.877332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f12c3-45df-41fd-b8d9-50f98cd20120 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:47.877623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f12c3-45df-41fd-b8d9-50f98cd20120 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:47.877898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f12c3-45df-41fd-b8d9-50f98cd20120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:47.894873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-963f12c3-45df-41fd-b8d9-50f98cd20120 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:47.896325 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-963f12c3-45df-41fd-b8d9-50f98cd20120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:47.933399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30958f-85f3-4253-b3b7-be3d5db86633 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:47.935124 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f30958f-85f3-4253-b3b7-be3d5db86633 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:47.976162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 451/911] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:47.994770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f12c3-45df-41fd-b8d9-50f98cd20120 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:48.009725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:48.044337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 461/912] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:47 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:38:48.057941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30958f-85f3-4253-b3b7-be3d5db86633 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:48.129466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f12c3-45df-41fd-b8d9-50f98cd20120 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:48.142368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:48.170947 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409522ee-c0dd-4a1a-9b2b-ec9f5ca46d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:48.171879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30958f-85f3-4253-b3b7-be3d5db86633 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:48.172391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409522ee-c0dd-4a1a-9b2b-ec9f5ca46d63 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:48.172995 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409522ee-c0dd-4a1a-9b2b-ec9f5ca46d63 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:48.173574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409522ee-c0dd-4a1a-9b2b-ec9f5ca46d63 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:48.174516 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409522ee-c0dd-4a1a-9b2b-ec9f5ca46d63 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:48.176134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409522ee-c0dd-4a1a-9b2b-ec9f5ca46d63 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-409522ee-c0dd-4a1a-9b2b-ec9f5ca46d63 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:48.190596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:48.191656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:48.192548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:48.192548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:48.192548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:48.192548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:48.192548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:48.192548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:48.192548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:48.192548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:48.192548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:48.192548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:48.192548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:48.192548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:48.192548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 17:38:48.192548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:48.195358 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 452/913] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:38:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:48.200907 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409522ee-c0dd-4a1a-9b2b-ec9f5ca46d63 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:48.217744 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674b37f-4717-41e1-bea6-1447ec112d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:48.218551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674b37f-4717-41e1-bea6-1447ec112d4c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:48.218833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674b37f-4717-41e1-bea6-1447ec112d4c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:48.219109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0674b37f-4717-41e1-bea6-1447ec112d4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:48.233620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674b37f-4717-41e1-bea6-1447ec112d4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:48.235728 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0674b37f-4717-41e1-bea6-1447ec112d4c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-0674b37f-4717-41e1-bea6-1447ec112d4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:48.261693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:48.262646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:48.263602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:48.263602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:48.263602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:48.263602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:48.263602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 462/914] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:48.269896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674b37f-4717-41e1-bea6-1447ec112d4c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:48.296889 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:48.326598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409522ee-c0dd-4a1a-9b2b-ec9f5ca46d63 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:48.362344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89aef7-2e9a-4646-82e3-e563f6bbc771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:48.363118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89aef7-2e9a-4646-82e3-e563f6bbc771 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:48.363469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89aef7-2e9a-4646-82e3-e563f6bbc771 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:48.363820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89aef7-2e9a-4646-82e3-e563f6bbc771 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:48.364580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89aef7-2e9a-4646-82e3-e563f6bbc771 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:48.364889 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89aef7-2e9a-4646-82e3-e563f6bbc771 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:48.366262 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89aef7-2e9a-4646-82e3-e563f6bbc771 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:48.371187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 453/915] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:38:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:48.372642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3daa78-ecbd-44f8-916a-17c3cd853a47 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:48.373495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3daa78-ecbd-44f8-916a-17c3cd853a47 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:48.373775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3daa78-ecbd-44f8-916a-17c3cd853a47 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:48.379408 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 454/916] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:48.380283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3daa78-ecbd-44f8-916a-17c3cd853a47 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:48.385440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0674b37f-4717-41e1-bea6-1447ec112d4c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:48.421538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce8bbd4-1a47-4e47-bacc-522eb0e9f47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:48.422201 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8bbd4-1a47-4e47-bacc-522eb0e9f47a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:48.422482 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8bbd4-1a47-4e47-bacc-522eb0e9f47a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:48.422754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8bbd4-1a47-4e47-bacc-522eb0e9f47a None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:48.423474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8bbd4-1a47-4e47-bacc-522eb0e9f47a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:48.423784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8bbd4-1a47-4e47-bacc-522eb0e9f47a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:48.423969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:48.425268 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce8bbd4-1a47-4e47-bacc-522eb0e9f47a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:48.442340 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 463/917] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:48.442340 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3208960-4690-486a-ba5f-20494a9d195b None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:48.442340 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3208960-4690-486a-ba5f-20494a9d195b None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:48.442340 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3208960-4690-486a-ba5f-20494a9d195b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:48.456906 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-f3208960-4690-486a-ba5f-20494a9d195b None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:48.458089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f3208960-4690-486a-ba5f-20494a9d195b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:48.500031 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3daa78-ecbd-44f8-916a-17c3cd853a47 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:48.532475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 464/918] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:48.533469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3208960-4690-486a-ba5f-20494a9d195b None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:48.535764 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945e8d5-94ed-4cab-9760-5a9e59b5dd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:48.536479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945e8d5-94ed-4cab-9760-5a9e59b5dd55 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:48.536767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945e8d5-94ed-4cab-9760-5a9e59b5dd55 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:48.537042 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945e8d5-94ed-4cab-9760-5a9e59b5dd55 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:48.537748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945e8d5-94ed-4cab-9760-5a9e59b5dd55 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:48.538055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945e8d5-94ed-4cab-9760-5a9e59b5dd55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:48.539415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945e8d5-94ed-4cab-9760-5a9e59b5dd55 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:48.550918 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 455/919] 10.222.0.64 () {56 vars in 1114 bytes} [Thu Jul 14 17:38:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:48.551770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945e8d5-94ed-4cab-9760-5a9e59b5dd55 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:48.554428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:48.645032 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3208960-4690-486a-ba5f-20494a9d195b None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:48.661967 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945e8d5-94ed-4cab-9760-5a9e59b5dd55 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:48.663756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:48.684657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed351d5-6718-44d8-9251-7c02f4b5279c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:48.686486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed351d5-6718-44d8-9251-7c02f4b5279c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:48.686784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed351d5-6718-44d8-9251-7c02f4b5279c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:48.687064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed351d5-6718-44d8-9251-7c02f4b5279c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:48.694034 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed351d5-6718-44d8-9251-7c02f4b5279c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:48.695473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed351d5-6718-44d8-9251-7c02f4b5279c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:48.700643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482320a-b49e-465a-bef7-1e6ab9583859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:48.701946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482320a-b49e-465a-bef7-1e6ab9583859 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:48.702232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482320a-b49e-465a-bef7-1e6ab9583859 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:48.702508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482320a-b49e-465a-bef7-1e6ab9583859 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:48.703245 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482320a-b49e-465a-bef7-1e6ab9583859 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:48.704635 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482320a-b49e-465a-bef7-1e6ab9583859 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-7ed351d5-6718-44d8-9251-7c02f4b5279c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:48.709737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:48.710640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:48.711536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:48.711536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:48.711536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:48.711536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:48.711536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:48.711536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:48.711536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:48.711536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:48.711536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 465/920] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:48.712770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed351d5-6718-44d8-9251-7c02f4b5279c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:48.743956 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 456/921] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Jul 14 17:38:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:38:48.743956 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482320a-b49e-465a-bef7-1e6ab9583859 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:48.847058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:48.878987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed351d5-6718-44d8-9251-7c02f4b5279c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:48.902193 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482320a-b49e-465a-bef7-1e6ab9583859 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:48.925113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb28a3-f3b8-4c12-9dbd-2da9b4e03c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:48.926653 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb28a3-f3b8-4c12-9dbd-2da9b4e03c95 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:48.927377 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb28a3-f3b8-4c12-9dbd-2da9b4e03c95 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:48.927377 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb28a3-f3b8-4c12-9dbd-2da9b4e03c95 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:48.927951 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb28a3-f3b8-4c12-9dbd-2da9b4e03c95 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:48.929329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eb28a3-f3b8-4c12-9dbd-2da9b4e03c95 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:48.945880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 466/922] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Jul 14 17:38:48 2022] POST /identity/v3/services => generated 241 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:38:48.962836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc41e6-0032-4f6b-93b7-ae683c3994b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:48.963533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc41e6-0032-4f6b-93b7-ae683c3994b1 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:48.963818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc41e6-0032-4f6b-93b7-ae683c3994b1 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:48.964157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc41e6-0032-4f6b-93b7-ae683c3994b1 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:48.964979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc41e6-0032-4f6b-93b7-ae683c3994b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:48.965296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc41e6-0032-4f6b-93b7-ae683c3994b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:48.966836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc41e6-0032-4f6b-93b7-ae683c3994b1 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:48.983852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 457/923] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:48.990285 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb28a3-f3b8-4c12-9dbd-2da9b4e03c95 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:49.116861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eb28a3-f3b8-4c12-9dbd-2da9b4e03c95 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:49.155279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccd09f-6d20-4a45-a953-ac789e87db2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:49.155956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccd09f-6d20-4a45-a953-ac789e87db2b None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:49.156242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccd09f-6d20-4a45-a953-ac789e87db2b None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:49.156515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccd09f-6d20-4a45-a953-ac789e87db2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:49.157351 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccd09f-6d20-4a45-a953-ac789e87db2b None admin] RBAC: Adding query filter params (role.id=086a0517b6e44f8aa33468ec2e3135a7, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=0cd1747822c3493e8aec150a12409fbd) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:49.157659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccd09f-6d20-4a45-a953-ac789e87db2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:49.159083 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccd09f-6d20-4a45-a953-ac789e87db2b None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:49.161064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:49.166094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 467/924] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:48 2022] GET /identity/v3/role_assignments?user.id=0cd1747822c3493e8aec150a12409fbd&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=086a0517b6e44f8aa33468ec2e3135a7 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:49.758606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a620f6-6062-491b-bb53-a0590d2cb498 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:49.758995 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a620f6-6062-491b-bb53-a0590d2cb498 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:49.759172 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a620f6-6062-491b-bb53-a0590d2cb498 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:49.760814 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 458/925] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:49.766554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a63a1f-e466-4227-bc97-63aefca20511 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:49.766840 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a63a1f-e466-4227-bc97-63aefca20511 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:49.767118 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a63a1f-e466-4227-bc97-63aefca20511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:49.784275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-05a63a1f-e466-4227-bc97-63aefca20511 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:49.785445 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-05a63a1f-e466-4227-bc97-63aefca20511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:49.851851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 468/926] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:49.875069 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a620f6-6062-491b-bb53-a0590d2cb498 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:49.887353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:49.987373 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a620f6-6062-491b-bb53-a0590d2cb498 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:50.022580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7d173-5d7c-4a53-8c5f-2ee4f8c192ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:50.023310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7d173-5d7c-4a53-8c5f-2ee4f8c192ed None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:50.023594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7d173-5d7c-4a53-8c5f-2ee4f8c192ed None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:50.023873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7d173-5d7c-4a53-8c5f-2ee4f8c192ed None admin] PATH_INFO: `/v3/services/compute` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:50.024549 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7d173-5d7c-4a53-8c5f-2ee4f8c192ed None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:50.025915 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7d173-5d7c-4a53-8c5f-2ee4f8c192ed None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-f7d7d173-5d7c-4a53-8c5f-2ee4f8c192ed None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:50.037697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:50.038727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:50.039563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:50.039563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:50.039563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:50.039563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:50.039563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:50.039563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:50.039563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:50.039563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:50.039563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:50.039563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:50.039563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:50.039563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:50.039563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 17:38:50.039563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:50.043918 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 459/927] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:50.045417 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a63a1f-e466-4227-bc97-63aefca20511 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:50.158552 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a63a1f-e466-4227-bc97-63aefca20511 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:50.188699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:50.196959 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3cdd4-72a5-4573-aaea-0b8c2063b729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:50.197697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3cdd4-72a5-4573-aaea-0b8c2063b729 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:50.197989 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3cdd4-72a5-4573-aaea-0b8c2063b729 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:50.198269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3cdd4-72a5-4573-aaea-0b8c2063b729 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:50.198993 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3cdd4-72a5-4573-aaea-0b8c2063b729 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:50.199304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3cdd4-72a5-4573-aaea-0b8c2063b729 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:50.200722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3cdd4-72a5-4573-aaea-0b8c2063b729 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:50.206659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 469/928] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Jul 14 17:38:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:50.212717 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7d173-5d7c-4a53-8c5f-2ee4f8c192ed None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:50.295751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:50.325185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7d173-5d7c-4a53-8c5f-2ee4f8c192ed None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:50.361235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458d5b9-3959-4acf-bc74-5c749eaada51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:50.361907 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458d5b9-3959-4acf-bc74-5c749eaada51 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:50.362187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458d5b9-3959-4acf-bc74-5c749eaada51 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:50.362462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458d5b9-3959-4acf-bc74-5c749eaada51 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:50.363203 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458d5b9-3959-4acf-bc74-5c749eaada51 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:50.363511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458d5b9-3959-4acf-bc74-5c749eaada51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:50.365236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458d5b9-3959-4acf-bc74-5c749eaada51 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:50.373787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 460/929] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Jul 14 17:38:50 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:50.906785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764afa3-be02-4964-ad1f-120ab3175e0f None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:50.907344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764afa3-be02-4964-ad1f-120ab3175e0f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:50.907344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f764afa3-be02-4964-ad1f-120ab3175e0f None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:50.909026 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 470/930] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:50.914280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d3231-64e7-41fe-92a4-3c9cffcb4334 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:50.914565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d3231-64e7-41fe-92a4-3c9cffcb4334 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:50.914840 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67d3231-64e7-41fe-92a4-3c9cffcb4334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:50.932239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f67d3231-64e7-41fe-92a4-3c9cffcb4334 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:50.933429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f67d3231-64e7-41fe-92a4-3c9cffcb4334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:51.008044 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 461/931] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:51.027185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764afa3-be02-4964-ad1f-120ab3175e0f None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:51.142298 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182b7ce-648f-4aa9-88f5-4e176a9c7bf4 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:51.142658 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182b7ce-648f-4aa9-88f5-4e176a9c7bf4 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:51.142854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182b7ce-648f-4aa9-88f5-4e176a9c7bf4 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:51.144644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 462/932] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:38:51.153063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ed9fc-49f9-4f43-81d4-7bf354d7218f None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:51.153521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ed9fc-49f9-4f43-81d4-7bf354d7218f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:51.154059 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ed9fc-49f9-4f43-81d4-7bf354d7218f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:51.155599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f764afa3-be02-4964-ad1f-120ab3175e0f None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:51.184627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-ba9ed9fc-49f9-4f43-81d4-7bf354d7218f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:51.187032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ba9ed9fc-49f9-4f43-81d4-7bf354d7218f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:51.203816 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f931a6-90c6-4226-a9be-95df4b2c0632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:51.204798 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f931a6-90c6-4226-a9be-95df4b2c0632 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:51.205242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f931a6-90c6-4226-a9be-95df4b2c0632 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:51.205893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f931a6-90c6-4226-a9be-95df4b2c0632 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:51.207248 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f931a6-90c6-4226-a9be-95df4b2c0632 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:51.209386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f931a6-90c6-4226-a9be-95df4b2c0632 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:51.211331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: WARNING keystone.server.flask.common [None req-b4f931a6-90c6-4226-a9be-95df4b2c0632 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 17:38:51.236704 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.sql.core [None req-b4f931a6-90c6-4226-a9be-95df4b2c0632 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 17:38:51.236704 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 17:38:51.236704 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [parameters: {'id': '94539f575f80403ba47184d3770205fe', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 17:38:51.236704 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91925) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 17:38:51.247178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-b4f931a6-90c6-4226-a9be-95df4b2c0632 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94539f575f80403ba47184d3770205fe.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94539f575f80403ba47184d3770205fe. Jul 14 17:38:51.247178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:51.247178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 17:38:51.247178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 17:38:51.247178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 17:38:51.247178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 17:38:51.247178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 17:38:51.247178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 17:38:51.247178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 17:38:51.247178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application conn.query(q) Jul 14 17:38:51.247178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 17:38:51.247178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 17:38:51.247178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application result.read() Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 14 17:38:51.248126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application next(self.gen) Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application yield resource Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application next(self.gen) Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise self.value Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application session.commit() Jul 14 17:38:51.249338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self.session.flush() Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application compat.raise_( Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 17:38:51.250229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 17:38:51.251121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application conn.query(q) Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application result.read() Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 17:38:51.252036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [parameters: {'id': '94539f575f80403ba47184d3770205fe', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:51.252920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:51.255879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:51.258772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:51.258772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 14 17:38:51.258772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 14 17:38:51.258772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:51.258772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:51.258772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 14 17:38:51.258772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 14 17:38:51.258772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 94539f575f80403ba47184d3770205fe. Jul 14 17:38:51.258772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:51.258772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 471/933] 10.222.0.64 () {60 vars in 1140 bytes} [Thu Jul 14 17:38:51 2022] POST /identity/v3/projects => generated 262 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:51.260165 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f931a6-90c6-4226-a9be-95df4b2c0632 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:51.285310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:51.294565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 463/934] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:51.296721 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8725-b33c-4103-b8a9-1c8e71cc66fa None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:51.297370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8725-b33c-4103-b8a9-1c8e71cc66fa None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:51.298286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd8725-b33c-4103-b8a9-1c8e71cc66fa None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:51.300614 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 464/935] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:51.309693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f0d0-f05a-4621-9be1-faf59db76c46 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:51.309977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f0d0-f05a-4621-9be1-faf59db76c46 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:51.310260 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474f0d0-f05a-4621-9be1-faf59db76c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:51.328075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:51.328366 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-c474f0d0-f05a-4621-9be1-faf59db76c46 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:51.329570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c474f0d0-f05a-4621-9be1-faf59db76c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:51.385597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f931a6-90c6-4226-a9be-95df4b2c0632 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:51.416104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 465/936] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:51.417411 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474f0d0-f05a-4621-9be1-faf59db76c46 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:51.420775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2a35c-4469-4b8a-939f-b296f2c9188c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:51.421816 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2a35c-4469-4b8a-939f-b296f2c9188c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:51.422298 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2a35c-4469-4b8a-939f-b296f2c9188c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:51.422857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2a35c-4469-4b8a-939f-b296f2c9188c None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:51.430610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2a35c-4469-4b8a-939f-b296f2c9188c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:51.432572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2a35c-4469-4b8a-939f-b296f2c9188c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-98a2a35c-4469-4b8a-939f-b296f2c9188c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:51.451030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:51.452272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:51.453229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:51.453229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:51.453229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:51.453229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:51.453229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:51.453229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:51.453229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:51.453229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:51.455791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 472/937] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:51.456859 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2a35c-4469-4b8a-939f-b296f2c9188c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:51.530129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474f0d0-f05a-4621-9be1-faf59db76c46 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:51.558345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:51.567562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23341ab6-da7c-4854-a79b-0440c0ae186d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:51.568407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23341ab6-da7c-4854-a79b-0440c0ae186d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:51.568690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23341ab6-da7c-4854-a79b-0440c0ae186d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:51.568973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23341ab6-da7c-4854-a79b-0440c0ae186d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:51.569633 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23341ab6-da7c-4854-a79b-0440c0ae186d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:51.570965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2a35c-4469-4b8a-939f-b296f2c9188c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:51.571070 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23341ab6-da7c-4854-a79b-0440c0ae186d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:51.580637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-23341ab6-da7c-4854-a79b-0440c0ae186d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:51.583349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:51.584381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:51.585207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 466/938] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Jul 14 17:38:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:51.585983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23341ab6-da7c-4854-a79b-0440c0ae186d None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:51.611974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6f10d-7c3a-4e48-a44c-fb5601b793c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:51.612802 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6f10d-7c3a-4e48-a44c-fb5601b793c4 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:51.613088 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6f10d-7c3a-4e48-a44c-fb5601b793c4 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:51.613368 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6f10d-7c3a-4e48-a44c-fb5601b793c4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:51.625303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b6f10d-7c3a-4e48-a44c-fb5601b793c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-32b6f10d-7c3a-4e48-a44c-fb5601b793c4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:51.636287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:51.637275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:51.638101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:51.638101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:51.638101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:51.638101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:51.638101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:51.638101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:51.638101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:51.638101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:51.638101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:51.638101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:51.638101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 17:38:51.638101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:51.639509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 473/939] 10.222.0.64 () {56 vars in 1120 bytes} [Thu Jul 14 17:38:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:51.644030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6f10d-7c3a-4e48-a44c-fb5601b793c4 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:51.699524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23341ab6-da7c-4854-a79b-0440c0ae186d None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:51.741520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa78541-633a-441c-b02f-46c732631ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:51.742097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa78541-633a-441c-b02f-46c732631ee1 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:51.742381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa78541-633a-441c-b02f-46c732631ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:51.742659 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa78541-633a-441c-b02f-46c732631ee1 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:51.743409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa78541-633a-441c-b02f-46c732631ee1 None admin] RBAC: Adding query filter params (name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:51.743717 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa78541-633a-441c-b02f-46c732631ee1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:51.745180 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa78541-633a-441c-b02f-46c732631ee1 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:51.764589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6f10d-7c3a-4e48-a44c-fb5601b793c4 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:51.773057 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 467/940] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Jul 14 17:38:51 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:51.774525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa78541-633a-441c-b02f-46c732631ee1 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:51.813939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf814cf5-9172-4abf-af58-b66fa0ee2331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:51.814934 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf814cf5-9172-4abf-af58-b66fa0ee2331 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:51.815659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf814cf5-9172-4abf-af58-b66fa0ee2331 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:51.816138 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf814cf5-9172-4abf-af58-b66fa0ee2331 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:51.817726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf814cf5-9172-4abf-af58-b66fa0ee2331 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:51.818321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf814cf5-9172-4abf-af58-b66fa0ee2331 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:51.820826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf814cf5-9172-4abf-af58-b66fa0ee2331 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:51.833186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 474/941] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Jul 14 17:38:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:51.843971 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf814cf5-9172-4abf-af58-b66fa0ee2331 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:51.851663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:51.880344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:51.910983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa78541-633a-441c-b02f-46c732631ee1 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:51.953467 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbcde0-84eb-4a5b-9045-4234701e5881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:51.954155 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbcde0-84eb-4a5b-9045-4234701e5881 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:51.954438 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbcde0-84eb-4a5b-9045-4234701e5881 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:51.954718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbcde0-84eb-4a5b-9045-4234701e5881 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:51.955474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbcde0-84eb-4a5b-9045-4234701e5881 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:51.955772 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbcde0-84eb-4a5b-9045-4234701e5881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:51.958117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbcde0-84eb-4a5b-9045-4234701e5881 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:51.958733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf814cf5-9172-4abf-af58-b66fa0ee2331 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:51.969232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 468/942] 10.222.0.64 () {56 vars in 1115 bytes} [Thu Jul 14 17:38:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:51.974756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbcde0-84eb-4a5b-9045-4234701e5881 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:52.004665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05266d-101d-4a8b-aab7-9d7a575b7dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:52.005395 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05266d-101d-4a8b-aab7-9d7a575b7dc0 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:52.005678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05266d-101d-4a8b-aab7-9d7a575b7dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:52.007006 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05266d-101d-4a8b-aab7-9d7a575b7dc0 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:52.007682 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05266d-101d-4a8b-aab7-9d7a575b7dc0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:52.008046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05266d-101d-4a8b-aab7-9d7a575b7dc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:52.009546 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05266d-101d-4a8b-aab7-9d7a575b7dc0 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:52.017108 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 475/943] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Jul 14 17:38:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:52.023071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05266d-101d-4a8b-aab7-9d7a575b7dc0 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:52.126012 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbcde0-84eb-4a5b-9045-4234701e5881 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:52.153269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:52.156488 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:52.165063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05266d-101d-4a8b-aab7-9d7a575b7dc0 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:52.198243 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8731384-1904-4c7c-ac39-89b141e4920a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:52.199754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8731384-1904-4c7c-ac39-89b141e4920a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:52.201067 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8731384-1904-4c7c-ac39-89b141e4920a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:52.202119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8731384-1904-4c7c-ac39-89b141e4920a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:52.216367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf10789-fd8b-4690-aebe-feff06e97d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:52.218068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf10789-fd8b-4690-aebe-feff06e97d15 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:52.218452 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf10789-fd8b-4690-aebe-feff06e97d15 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:52.218803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf10789-fd8b-4690-aebe-feff06e97d15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:52.220266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf10789-fd8b-4690-aebe-feff06e97d15 None admin] RBAC: Adding query filter params (interface=public, service_id=f7a1f35ec9b04b9496365925a3b14fa8, region_id=RegionOne) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:52.220782 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf10789-fd8b-4690-aebe-feff06e97d15 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:52.221841 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8731384-1904-4c7c-ac39-89b141e4920a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:52.222791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf10789-fd8b-4690-aebe-feff06e97d15 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:52.223809 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8731384-1904-4c7c-ac39-89b141e4920a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:52.230191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 476/944] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Jul 14 17:38:52 2022] GET /identity/v3/endpoints?service_id=f7a1f35ec9b04b9496365925a3b14fa8&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:52.237575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf10789-fd8b-4690-aebe-feff06e97d15 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-c8731384-1904-4c7c-ac39-89b141e4920a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:52.247362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:52.248820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:52.249705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:52.249705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:52.249705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:52.249705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:52.251715 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 469/945] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:52.257682 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8731384-1904-4c7c-ac39-89b141e4920a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:52.377749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8731384-1904-4c7c-ac39-89b141e4920a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:52.386779 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf10789-fd8b-4690-aebe-feff06e97d15 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:52.415116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda9dfe-fdf7-40c5-9233-51c7b6ac6169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:52.415582 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda9dfe-fdf7-40c5-9233-51c7b6ac6169 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:52.415817 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda9dfe-fdf7-40c5-9233-51c7b6ac6169 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:52.416132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bda9dfe-fdf7-40c5-9233-51c7b6ac6169 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:52.416881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda9dfe-fdf7-40c5-9233-51c7b6ac6169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:52.417186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda9dfe-fdf7-40c5-9233-51c7b6ac6169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:52.418929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bda9dfe-fdf7-40c5-9233-51c7b6ac6169 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:52.427979 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e307e778-2016-4c30-9d52-8966ef5296f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:52.428917 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307e778-2016-4c30-9d52-8966ef5296f7 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:52.429371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307e778-2016-4c30-9d52-8966ef5296f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:52.429795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e307e778-2016-4c30-9d52-8966ef5296f7 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:52.430648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307e778-2016-4c30-9d52-8966ef5296f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:52.432355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e307e778-2016-4c30-9d52-8966ef5296f7 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:52.432982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 470/946] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:52.439238 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda9dfe-fdf7-40c5-9233-51c7b6ac6169 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:52.441478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 477/947] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Jul 14 17:38:52 2022] GET /identity/v3/services => generated 1497 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:38:52.474943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:52.515726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0a85d-7a9a-42ed-b55d-7fe6645e3768 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:52.516286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0a85d-7a9a-42ed-b55d-7fe6645e3768 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:52.516770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0a85d-7a9a-42ed-b55d-7fe6645e3768 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:52.518745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 478/948] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:52.525733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948befb-c406-4e06-802d-e85502152da9 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:52.526317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948befb-c406-4e06-802d-e85502152da9 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:52.526597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948befb-c406-4e06-802d-e85502152da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:52.546376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-3948befb-c406-4e06-802d-e85502152da9 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:52.548025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3948befb-c406-4e06-802d-e85502152da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:52.559816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bda9dfe-fdf7-40c5-9233-51c7b6ac6169 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:52.568483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:52.604062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaec8d-1246-4e07-a549-3eaf2cf4c408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:52.604909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaec8d-1246-4e07-a549-3eaf2cf4c408 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:52.605197 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaec8d-1246-4e07-a549-3eaf2cf4c408 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:52.605480 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaec8d-1246-4e07-a549-3eaf2cf4c408 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:52.614788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abaec8d-1246-4e07-a549-3eaf2cf4c408 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:52.616468 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abaec8d-1246-4e07-a549-3eaf2cf4c408 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:52.633408 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 479/949] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-9abaec8d-1246-4e07-a549-3eaf2cf4c408 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:52.635522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:52.636660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:52.637806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:52.637806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:52.637806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:52.637806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:52.637806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:52.637806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:52.637806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:52.637806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:52.637806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 17:38:52.637806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:52.637806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 471/950] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Jul 14 17:38:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:52.642783 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948befb-c406-4e06-802d-e85502152da9 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:52.651024 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaec8d-1246-4e07-a549-3eaf2cf4c408 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:52.761293 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:52.766796 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948befb-c406-4e06-802d-e85502152da9 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:52.781585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaec8d-1246-4e07-a549-3eaf2cf4c408 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:52.805470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b32777-2cbd-44d2-9ce1-5a6fd94e3de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:52.806155 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b32777-2cbd-44d2-9ce1-5a6fd94e3de5 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:52.806443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b32777-2cbd-44d2-9ce1-5a6fd94e3de5 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:52.806722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b32777-2cbd-44d2-9ce1-5a6fd94e3de5 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:52.807569 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b32777-2cbd-44d2-9ce1-5a6fd94e3de5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:52.807882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b32777-2cbd-44d2-9ce1-5a6fd94e3de5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:52.809480 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b32777-2cbd-44d2-9ce1-5a6fd94e3de5 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:52.826582 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1307c-739a-404b-a88c-58deabd98fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:52.827524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1307c-739a-404b-a88c-58deabd98fa0 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:52.827942 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1307c-739a-404b-a88c-58deabd98fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:52.828232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1307c-739a-404b-a88c-58deabd98fa0 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:52.828970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b1307c-739a-404b-a88c-58deabd98fa0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:52.830345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b1307c-739a-404b-a88c-58deabd98fa0 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:52.834735 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 480/951] 10.222.0.64 () {56 vars in 1149 bytes} [Thu Jul 14 17:38:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:52.835993 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b32777-2cbd-44d2-9ce1-5a6fd94e3de5 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:52.852295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 472/952] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Jul 14 17:38:52 2022] POST /identity/v3/services => generated 242 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:38:52.865644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:52.972049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b32777-2cbd-44d2-9ce1-5a6fd94e3de5 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:53.020590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e407e-3cb4-423f-ba3d-86a03a1e88fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:53.021685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e407e-3cb4-423f-ba3d-86a03a1e88fc None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:53.022113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e407e-3cb4-423f-ba3d-86a03a1e88fc None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:53.022562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e407e-3cb4-423f-ba3d-86a03a1e88fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:53.035950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e407e-3cb4-423f-ba3d-86a03a1e88fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:53.035950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e407e-3cb4-423f-ba3d-86a03a1e88fc None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-315e407e-3cb4-423f-ba3d-86a03a1e88fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:53.055655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:53.056794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:53.057967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:53.057967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:53.057967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:53.057967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:53.057967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 481/953] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:53.063471 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1307c-739a-404b-a88c-58deabd98fa0 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:53.174063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1307c-739a-404b-a88c-58deabd98fa0 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:53.210034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934df33-aa9e-48f7-83cc-b0c1196c3e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:53.210710 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934df33-aa9e-48f7-83cc-b0c1196c3e95 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:53.210999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934df33-aa9e-48f7-83cc-b0c1196c3e95 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:53.211276 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4934df33-aa9e-48f7-83cc-b0c1196c3e95 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:53.212029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934df33-aa9e-48f7-83cc-b0c1196c3e95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:53.212345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934df33-aa9e-48f7-83cc-b0c1196c3e95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:53.213714 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4934df33-aa9e-48f7-83cc-b0c1196c3e95 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:53.226063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 473/954] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:53.231847 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e407e-3cb4-423f-ba3d-86a03a1e88fc None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:53.294745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:53.336167 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e407e-3cb4-423f-ba3d-86a03a1e88fc None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:53.375233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229632dc-8338-42ec-95e3-af851d3481a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:53.376071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229632dc-8338-42ec-95e3-af851d3481a7 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:53.376378 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229632dc-8338-42ec-95e3-af851d3481a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:53.376618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229632dc-8338-42ec-95e3-af851d3481a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:53.383767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229632dc-8338-42ec-95e3-af851d3481a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:53.385229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229632dc-8338-42ec-95e3-af851d3481a7 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-229632dc-8338-42ec-95e3-af851d3481a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:53.399899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:53.402132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:53.404383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:53.404383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:53.404383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:53.404383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:53.404383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:53.404383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:53.404383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:53.404383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:53.404383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 482/955] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:53.408081 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934df33-aa9e-48f7-83cc-b0c1196c3e95 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:53.416052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:53.520380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4934df33-aa9e-48f7-83cc-b0c1196c3e95 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:53.557823 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832cb789-d6db-4b5a-b5a7-2e1bca7c6a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:53.558412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cb789-d6db-4b5a-b5a7-2e1bca7c6a7a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:53.558697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cb789-d6db-4b5a-b5a7-2e1bca7c6a7a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:53.558978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832cb789-d6db-4b5a-b5a7-2e1bca7c6a7a None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:53.559768 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cb789-d6db-4b5a-b5a7-2e1bca7c6a7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:53.560117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cb789-d6db-4b5a-b5a7-2e1bca7c6a7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:53.561580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832cb789-d6db-4b5a-b5a7-2e1bca7c6a7a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:53.574138 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 474/956] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:53.580611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229632dc-8338-42ec-95e3-af851d3481a7 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:53.692918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229632dc-8338-42ec-95e3-af851d3481a7 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:53.713542 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:53.729390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255729b-1c74-449b-87b0-2736483ec34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:53.730287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255729b-1c74-449b-87b0-2736483ec34c None admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:53.730571 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255729b-1c74-449b-87b0-2736483ec34c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:53.730830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255729b-1c74-449b-87b0-2736483ec34c None admin] PATH_INFO: `/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/0cd1747822c3493e8aec150a12409fbd/roles/086a0517b6e44f8aa33468ec2e3135a7` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:53.733686 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255729b-1c74-449b-87b0-2736483ec34c None admin] RBAC: Authorizing `identity:create_grant(project_id=de62d2fb17f846f2bce5bef108353fd8, user_id=0cd1747822c3493e8aec150a12409fbd, role_id=086a0517b6e44f8aa33468ec2e3135a7)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:53.735127 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0255729b-1c74-449b-87b0-2736483ec34c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:53.751339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 483/957] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:38:53 2022] PUT /identity/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/0cd1747822c3493e8aec150a12409fbd/roles/086a0517b6e44f8aa33468ec2e3135a7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:38:54.057084 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7423f-33b6-43a1-b036-c522697a4e8a None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:54.057469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7423f-33b6-43a1-b036-c522697a4e8a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:54.057657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7423f-33b6-43a1-b036-c522697a4e8a None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:54.059233 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 475/958] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:54.065348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793170d-f078-4bca-a9e7-b5e119a6f950 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:54.065862 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793170d-f078-4bca-a9e7-b5e119a6f950 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:54.065941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793170d-f078-4bca-a9e7-b5e119a6f950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:54.084072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-6793170d-f078-4bca-a9e7-b5e119a6f950 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:54.085264 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6793170d-f078-4bca-a9e7-b5e119a6f950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:54.160371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 484/959] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:54.186202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7423f-33b6-43a1-b036-c522697a4e8a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:54.259793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:54.303251 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7423f-33b6-43a1-b036-c522697a4e8a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:54.339753 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e68d5-0585-4c1d-8fba-234d21e1ece6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:54.340844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e68d5-0585-4c1d-8fba-234d21e1ece6 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:54.341352 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e68d5-0585-4c1d-8fba-234d21e1ece6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:54.341873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e68d5-0585-4c1d-8fba-234d21e1ece6 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:54.343300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e68d5-0585-4c1d-8fba-234d21e1ece6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:54.344960 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e68d5-0585-4c1d-8fba-234d21e1ece6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:54.346565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: WARNING keystone.server.flask.common [None req-5e6e68d5-0585-4c1d-8fba-234d21e1ece6 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 17:38:54.359729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.sql.core [None req-5e6e68d5-0585-4c1d-8fba-234d21e1ece6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 17:38:54.359729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 17:38:54.359729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [parameters: {'user_id': 'a78d467cc1604033a78ceaca0def55f4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 17:38:54.359729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=91926) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-5e6e68d5-0585-4c1d-8fba-234d21e1ece6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application conn.query(q) Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application result.read() Jul 14 17:38:54.379133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 17:38:54.380130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application next(self.gen) Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application yield resource Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application next(self.gen) Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise self.value Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application session.commit() Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 17:38:54.381257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self.session.flush() Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application compat.raise_( Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 17:38:54.382174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 17:38:54.383673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application conn.query(q) Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application result.read() Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 17:38:54.384594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a78d467cc1604033a78ceaca0def55f4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:54.386000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 14 17:38:54.387991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 14 17:38:54.390697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 14 17:38:54.390697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 14 17:38:54.390697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 17:38:54.390697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 17:38:54.390697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 17:38:54.390697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:54.393387 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 476/960] 10.222.0.64 () {60 vars in 1131 bytes} [Thu Jul 14 17:38:54 2022] POST /identity/v3/users => generated 160 bytes in 208 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:54.402553 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793170d-f078-4bca-a9e7-b5e119a6f950 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:54.507206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793170d-f078-4bca-a9e7-b5e119a6f950 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:54.518848 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:54.544003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610a9723-151e-4b24-8a4b-731567bc21b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:54.544903 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a9723-151e-4b24-8a4b-731567bc21b3 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:54.545799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a9723-151e-4b24-8a4b-731567bc21b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:54.546144 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610a9723-151e-4b24-8a4b-731567bc21b3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:54.555250 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a9723-151e-4b24-8a4b-731567bc21b3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:54.557343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610a9723-151e-4b24-8a4b-731567bc21b3 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-610a9723-151e-4b24-8a4b-731567bc21b3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:54.573788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:54.576107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:54.578436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:54.578436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:54.578436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:54.578436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:54.578436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:54.578436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:54.578436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:54.578436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:54.578436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 17:38:54.578436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:54.578436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 485/961] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:38:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:54.585427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e68d5-0585-4c1d-8fba-234d21e1ece6 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:54.616147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a1aaf-c471-4a1b-b64c-51237068ecfa None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:54.616323 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a1aaf-c471-4a1b-b64c-51237068ecfa None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:54.616586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a1aaf-c471-4a1b-b64c-51237068ecfa None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:54.621939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 486/962] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:38:54.624183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135761-c540-4ca8-916d-ef95248412c0 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:54.624469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135761-c540-4ca8-916d-ef95248412c0 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:54.624743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55135761-c540-4ca8-916d-ef95248412c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:54.642062 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-55135761-c540-4ca8-916d-ef95248412c0 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:54.643244 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-55135761-c540-4ca8-916d-ef95248412c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:54.710242 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e68d5-0585-4c1d-8fba-234d21e1ece6 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:54.722592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 487/963] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:38:54.746650 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55135761-c540-4ca8-916d-ef95248412c0 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:54.756154 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:54.767423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33270778-eb42-41ae-a5fa-d710b745d70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:54.768451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33270778-eb42-41ae-a5fa-d710b745d70c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:54.768738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33270778-eb42-41ae-a5fa-d710b745d70c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:54.769014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33270778-eb42-41ae-a5fa-d710b745d70c None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:54.769826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33270778-eb42-41ae-a5fa-d710b745d70c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:54.770132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33270778-eb42-41ae-a5fa-d710b745d70c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:54.772348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33270778-eb42-41ae-a5fa-d710b745d70c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:54.823452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 477/964] 10.222.0.64 () {56 vars in 1101 bytes} [Thu Jul 14 17:38:54 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:54.876259 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:54.880834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55135761-c540-4ca8-916d-ef95248412c0 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:54.923431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60eaca9-9d1c-4c53-a60e-cc13d9bbc620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:54.924719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60eaca9-9d1c-4c53-a60e-cc13d9bbc620 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:54.925233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60eaca9-9d1c-4c53-a60e-cc13d9bbc620 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:54.925724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60eaca9-9d1c-4c53-a60e-cc13d9bbc620 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:54.926640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60eaca9-9d1c-4c53-a60e-cc13d9bbc620 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:54.928441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60eaca9-9d1c-4c53-a60e-cc13d9bbc620 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-e60eaca9-9d1c-4c53-a60e-cc13d9bbc620 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:54.943194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:54.944393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:54.945283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:54.945283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:54.945283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:54.945283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:54.945283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:54.945283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:54.945283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:54.945283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:54.945283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:54.945283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:54.945283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:54.945283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:54.945283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:38:54.945283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:54.948522 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 488/965] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Jul 14 17:38:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:54.956095 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33270778-eb42-41ae-a5fa-d710b745d70c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:55.069962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33270778-eb42-41ae-a5fa-d710b745d70c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:55.105431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e6fb51-49ed-4d4e-bcb3-af8d7a49bfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:55.106107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6fb51-49ed-4d4e-bcb3-af8d7a49bfa5 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:55.106394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6fb51-49ed-4d4e-bcb3-af8d7a49bfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:55.106673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e6fb51-49ed-4d4e-bcb3-af8d7a49bfa5 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:55.107387 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6fb51-49ed-4d4e-bcb3-af8d7a49bfa5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:55.108242 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6fb51-49ed-4d4e-bcb3-af8d7a49bfa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:55.109618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e6fb51-49ed-4d4e-bcb3-af8d7a49bfa5 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:55.114950 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 478/966] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Jul 14 17:38:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:55.120666 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60eaca9-9d1c-4c53-a60e-cc13d9bbc620 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:55.124578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0db078-6186-435e-8757-ac00c6f56278 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:55.124869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0db078-6186-435e-8757-ac00c6f56278 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:55.125151 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0db078-6186-435e-8757-ac00c6f56278 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:55.126737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 479/967] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:38:55.131363 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649ad0b-acc3-45ba-b750-69d432fa5b40 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:55.131657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649ad0b-acc3-45ba-b750-69d432fa5b40 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:55.131973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649ad0b-acc3-45ba-b750-69d432fa5b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:55.150100 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-c649ad0b-acc3-45ba-b750-69d432fa5b40 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:55.151611 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c649ad0b-acc3-45ba-b750-69d432fa5b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:55.202012 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:55.228464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 480/968] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:38:55.235635 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60eaca9-9d1c-4c53-a60e-cc13d9bbc620 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:55.251435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649ad0b-acc3-45ba-b750-69d432fa5b40 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:55.273372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dabe1-d413-488f-aca8-dd55017dd37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:55.274045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dabe1-d413-488f-aca8-dd55017dd37c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:55.274324 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dabe1-d413-488f-aca8-dd55017dd37c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:55.274599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18dabe1-d413-488f-aca8-dd55017dd37c None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:55.275302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dabe1-d413-488f-aca8-dd55017dd37c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:55.275607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dabe1-d413-488f-aca8-dd55017dd37c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:55.277018 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18dabe1-d413-488f-aca8-dd55017dd37c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:55.282181 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 489/969] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Jul 14 17:38:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:55.288585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dabe1-d413-488f-aca8-dd55017dd37c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:55.319485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:55.361778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649ad0b-acc3-45ba-b750-69d432fa5b40 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:55.393857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18dabe1-d413-488f-aca8-dd55017dd37c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:55.397760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036561c-2ba6-4157-be30-79920ae3b7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:55.398563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036561c-2ba6-4157-be30-79920ae3b7f3 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:55.398844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036561c-2ba6-4157-be30-79920ae3b7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:55.399117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036561c-2ba6-4157-be30-79920ae3b7f3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:55.399763 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036561c-2ba6-4157-be30-79920ae3b7f3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:55.401381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036561c-2ba6-4157-be30-79920ae3b7f3 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-2036561c-2ba6-4157-be30-79920ae3b7f3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:55.414581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:55.415617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:55.416520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 481/970] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:55.422597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036561c-2ba6-4157-be30-79920ae3b7f3 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:55.431738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5965a8-ebf4-42ef-8c3d-b71127019666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:55.432456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5965a8-ebf4-42ef-8c3d-b71127019666 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:55.432742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5965a8-ebf4-42ef-8c3d-b71127019666 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:55.433019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5965a8-ebf4-42ef-8c3d-b71127019666 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:55.433765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5965a8-ebf4-42ef-8c3d-b71127019666 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:55.435126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5965a8-ebf4-42ef-8c3d-b71127019666 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:55.467962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 490/971] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Jul 14 17:38:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:38:55.488641 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:55.552272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036561c-2ba6-4157-be30-79920ae3b7f3 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:55.597188 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eaf21e-2a72-4c7f-a86d-096ccd26b23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:55.597870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eaf21e-2a72-4c7f-a86d-096ccd26b23e None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:55.598152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eaf21e-2a72-4c7f-a86d-096ccd26b23e None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:55.598429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eaf21e-2a72-4c7f-a86d-096ccd26b23e None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:55.599146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eaf21e-2a72-4c7f-a86d-096ccd26b23e None admin] RBAC: Adding query filter params (name=compute) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:55.599456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eaf21e-2a72-4c7f-a86d-096ccd26b23e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:55.600861 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eaf21e-2a72-4c7f-a86d-096ccd26b23e None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:55.607546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 482/972] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Jul 14 17:38:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:55.613235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5965a8-ebf4-42ef-8c3d-b71127019666 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:55.708938 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:55.731676 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5965a8-ebf4-42ef-8c3d-b71127019666 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:55.769698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae71971-24e9-4dd2-8a57-119f7c22fec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:55.770383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae71971-24e9-4dd2-8a57-119f7c22fec3 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:55.770670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae71971-24e9-4dd2-8a57-119f7c22fec3 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:55.770949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae71971-24e9-4dd2-8a57-119f7c22fec3 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:55.771677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae71971-24e9-4dd2-8a57-119f7c22fec3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:55.772014 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae71971-24e9-4dd2-8a57-119f7c22fec3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:55.773418 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae71971-24e9-4dd2-8a57-119f7c22fec3 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:55.784458 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 491/973] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Jul 14 17:38:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:55.785913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eaf21e-2a72-4c7f-a86d-096ccd26b23e None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:55.891878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eaf21e-2a72-4c7f-a86d-096ccd26b23e None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:55.894247 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:55.928066 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c45d3af-f35d-4f83-ab89-914a2165d173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:55.928736 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45d3af-f35d-4f83-ab89-914a2165d173 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:55.929022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45d3af-f35d-4f83-ab89-914a2165d173 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:55.929303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c45d3af-f35d-4f83-ab89-914a2165d173 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:55.930089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45d3af-f35d-4f83-ab89-914a2165d173 None admin] RBAC: Adding query filter params (interface=public, service_id=b359a725f9cc4ed987b7b601b7aebb50, region_id=RegionOne) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:55.930406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45d3af-f35d-4f83-ab89-914a2165d173 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:55.931770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c45d3af-f35d-4f83-ab89-914a2165d173 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:55.938983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 483/974] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Jul 14 17:38:55 2022] GET /identity/v3/endpoints?service_id=b359a725f9cc4ed987b7b601b7aebb50&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:55.943225 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae71971-24e9-4dd2-8a57-119f7c22fec3 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:55.972195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f04db8-5698-4de3-bbbf-1de85f286dbb None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:55.972625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f04db8-5698-4de3-bbbf-1de85f286dbb None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:55.973125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f04db8-5698-4de3-bbbf-1de85f286dbb None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:55.975888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 484/975] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:55.980926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c88bb6-9463-4aca-ac38-21c875bad9e3 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:55.981222 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c88bb6-9463-4aca-ac38-21c875bad9e3 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:55.981509 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c88bb6-9463-4aca-ac38-21c875bad9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:55.999346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-c5c88bb6-9463-4aca-ac38-21c875bad9e3 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:56.000581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c5c88bb6-9463-4aca-ac38-21c875bad9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:56.069191 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 485/976] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:55 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:38:56.095425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c88bb6-9463-4aca-ac38-21c875bad9e3 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:56.099764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae71971-24e9-4dd2-8a57-119f7c22fec3 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:56.147758 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe580f6-f210-4316-8d3f-b1fd981cce63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:56.149241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe580f6-f210-4316-8d3f-b1fd981cce63 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:56.149757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe580f6-f210-4316-8d3f-b1fd981cce63 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:56.150253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe580f6-f210-4316-8d3f-b1fd981cce63 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:56.151140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe580f6-f210-4316-8d3f-b1fd981cce63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:56.152791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe580f6-f210-4316-8d3f-b1fd981cce63 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:56.168423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 492/977] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Jul 14 17:38:55 2022] GET /identity/v3/services => generated 1727 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:38:56.206282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:56.210640 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c88bb6-9463-4aca-ac38-21c875bad9e3 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:56.251898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629e98a-57e6-4406-a020-581dd02086c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:56.252690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629e98a-57e6-4406-a020-581dd02086c3 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:56.252981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629e98a-57e6-4406-a020-581dd02086c3 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:56.253264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c629e98a-57e6-4406-a020-581dd02086c3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:56.265476 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c629e98a-57e6-4406-a020-581dd02086c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-c629e98a-57e6-4406-a020-581dd02086c3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:38:56.274007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:56.275077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:56.281958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:56.281958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:56.281958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:56.281958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:56.281958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:38:56.281958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:38:56.281958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:38:56.281958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:38:56.281958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:38:56.281958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:38:56.281958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 17:38:56.281958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:56.281958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 486/978] 10.222.0.64 () {56 vars in 1120 bytes} [Thu Jul 14 17:38:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:56.284562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe580f6-f210-4316-8d3f-b1fd981cce63 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:56.301675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:56.395967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe580f6-f210-4316-8d3f-b1fd981cce63 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:56.430580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9761b5d-70ec-4dba-aa93-3be24e63960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:56.431252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9761b5d-70ec-4dba-aa93-3be24e63960c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:56.431534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9761b5d-70ec-4dba-aa93-3be24e63960c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:56.431809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9761b5d-70ec-4dba-aa93-3be24e63960c None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:56.432604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9761b5d-70ec-4dba-aa93-3be24e63960c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:56.432915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9761b5d-70ec-4dba-aa93-3be24e63960c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:56.434294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9761b5d-70ec-4dba-aa93-3be24e63960c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:56.442684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 493/979] 10.222.0.64 () {56 vars in 1115 bytes} [Thu Jul 14 17:38:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:56.448245 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629e98a-57e6-4406-a020-581dd02086c3 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:56.552445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c629e98a-57e6-4406-a020-581dd02086c3 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:56.585647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24d2c0-0267-4f47-aa4d-22887386d34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:56.586447 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24d2c0-0267-4f47-aa4d-22887386d34b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:56.586729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24d2c0-0267-4f47-aa4d-22887386d34b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:56.587000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24d2c0-0267-4f47-aa4d-22887386d34b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:56.598255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24d2c0-0267-4f47-aa4d-22887386d34b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:56.599692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd24d2c0-0267-4f47-aa4d-22887386d34b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-dd24d2c0-0267-4f47-aa4d-22887386d34b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:56.617286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:56.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:56.619170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:56.619170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:56.619170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:56.619170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:56.624254 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 487/980] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:56.625700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9761b5d-70ec-4dba-aa93-3be24e63960c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:56.704685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:56.731150 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9761b5d-70ec-4dba-aa93-3be24e63960c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:56.765785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a868b-81ad-47e1-9867-cd86d32a2d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:56.766448 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a868b-81ad-47e1-9867-cd86d32a2d2a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:56.766731 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a868b-81ad-47e1-9867-cd86d32a2d2a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:56.767009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a868b-81ad-47e1-9867-cd86d32a2d2a None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:56.767732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a868b-81ad-47e1-9867-cd86d32a2d2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:56.768196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a868b-81ad-47e1-9867-cd86d32a2d2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:56.769539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a868b-81ad-47e1-9867-cd86d32a2d2a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:56.785996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 494/981] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:56.787595 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24d2c0-0267-4f47-aa4d-22887386d34b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:56.801530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:56.894458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24d2c0-0267-4f47-aa4d-22887386d34b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:56.928763 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dc994-895e-431b-8a20-bda2a59d2634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:56.929576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dc994-895e-431b-8a20-bda2a59d2634 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:56.929858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dc994-895e-431b-8a20-bda2a59d2634 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:56.930132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dc994-895e-431b-8a20-bda2a59d2634 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:56.938535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dc994-895e-431b-8a20-bda2a59d2634 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:56.940030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dc994-895e-431b-8a20-bda2a59d2634 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-5e1dc994-895e-431b-8a20-bda2a59d2634 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:38:56.955266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:56.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:56.957462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:56.957462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:56.957462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:38:56.957462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:38:56.957462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:38:56.957462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:38:56.957462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:38:56.957462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:38:56.957462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 17:38:56.957462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:56.960250 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 488/982] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Jul 14 17:38:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:56.962215 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a868b-81ad-47e1-9867-cd86d32a2d2a None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:57.060830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a868b-81ad-47e1-9867-cd86d32a2d2a None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:57.099576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efe2c8-8fbe-4196-9d96-5edcbeaf069b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:57.100276 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efe2c8-8fbe-4196-9d96-5edcbeaf069b None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:57.100558 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efe2c8-8fbe-4196-9d96-5edcbeaf069b None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:57.100828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efe2c8-8fbe-4196-9d96-5edcbeaf069b None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:57.101668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efe2c8-8fbe-4196-9d96-5edcbeaf069b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:57.101972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efe2c8-8fbe-4196-9d96-5edcbeaf069b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:57.103383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efe2c8-8fbe-4196-9d96-5edcbeaf069b None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:57.105173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:57.133283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 495/983] 10.222.0.64 () {56 vars in 1149 bytes} [Thu Jul 14 17:38:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:57.134733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dc994-895e-431b-8a20-bda2a59d2634 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:57.208217 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:57.241302 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dc994-895e-431b-8a20-bda2a59d2634 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:57.281626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7fcc2-24c4-4b33-aed9-db81dc09f5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:57.282286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7fcc2-24c4-4b33-aed9-db81dc09f5a4 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:57.282569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7fcc2-24c4-4b33-aed9-db81dc09f5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:57.282934 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7fcc2-24c4-4b33-aed9-db81dc09f5a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:57.352835 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7fcc2-24c4-4b33-aed9-db81dc09f5a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:57.354305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7fcc2-24c4-4b33-aed9-db81dc09f5a4 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:57.429415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:57.463163 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 489/984] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:38:57 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:38:57.469011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efe2c8-8fbe-4196-9d96-5edcbeaf069b None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:57.567718 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efe2c8-8fbe-4196-9d96-5edcbeaf069b None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:57.572585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d69eca-a686-464c-9e0a-5293ea5f88d1 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:57.572811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d69eca-a686-464c-9e0a-5293ea5f88d1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:57.573075 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d69eca-a686-464c-9e0a-5293ea5f88d1 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:57.574591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 490/985] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:38:57.579445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4abad-18bb-4260-a64e-06e8c3f6a98b None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:57.579741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4abad-18bb-4260-a64e-06e8c3f6a98b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:57.580062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4abad-18bb-4260-a64e-06e8c3f6a98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:57.597357 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-acb4abad-18bb-4260-a64e-06e8c3f6a98b None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:57.598543 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-acb4abad-18bb-4260-a64e-06e8c3f6a98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:57.604687 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fac487-53e8-45b0-bcca-96a53a81dae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:57.605482 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fac487-53e8-45b0-bcca-96a53a81dae4 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:57.605770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fac487-53e8-45b0-bcca-96a53a81dae4 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:57.606054 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fac487-53e8-45b0-bcca-96a53a81dae4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:57.617275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fac487-53e8-45b0-bcca-96a53a81dae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:57.618712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fac487-53e8-45b0-bcca-96a53a81dae4 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-61fac487-53e8-45b0-bcca-96a53a81dae4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:38:57.637528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:57.638493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:57.639544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:38:57.639544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:38:57.639544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:38:57.639544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:57.643621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 496/986] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:57.644522 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fac487-53e8-45b0-bcca-96a53a81dae4 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:57.671452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 491/987] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:38:57.691613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:57.694182 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4abad-18bb-4260-a64e-06e8c3f6a98b None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:57.751574 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fac487-53e8-45b0-bcca-96a53a81dae4 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:57.785983 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669dc720-c4d7-4461-a8c7-37efe1f573be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:57.786655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dc720-c4d7-4461-a8c7-37efe1f573be None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:57.786942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dc720-c4d7-4461-a8c7-37efe1f573be None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:57.787223 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dc720-c4d7-4461-a8c7-37efe1f573be None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:57.787977 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669dc720-c4d7-4461-a8c7-37efe1f573be None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:57.788429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669dc720-c4d7-4461-a8c7-37efe1f573be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:57.789675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669dc720-c4d7-4461-a8c7-37efe1f573be None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:57.800530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:57.802335 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 497/988] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:57.805278 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4abad-18bb-4260-a64e-06e8c3f6a98b None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:57.808038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669dc720-c4d7-4461-a8c7-37efe1f573be None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:57.845602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226da95-98d0-45d4-aeea-9ce2c2b718c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:57.846648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226da95-98d0-45d4-aeea-9ce2c2b718c9 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:57.846931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226da95-98d0-45d4-aeea-9ce2c2b718c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:57.847229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6226da95-98d0-45d4-aeea-9ce2c2b718c9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:57.847886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226da95-98d0-45d4-aeea-9ce2c2b718c9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:57.849350 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6226da95-98d0-45d4-aeea-9ce2c2b718c9 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-6226da95-98d0-45d4-aeea-9ce2c2b718c9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:57.863255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:57.865374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:38:57.866963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 492/989] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Jul 14 17:38:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:57.870640 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226da95-98d0-45d4-aeea-9ce2c2b718c9 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:57.923381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669dc720-c4d7-4461-a8c7-37efe1f573be None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:57.925908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:57.963107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5405f10-458f-46e0-a9a9-82de8e5d8d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:57.964027 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5405f10-458f-46e0-a9a9-82de8e5d8d09 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:57.964390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5405f10-458f-46e0-a9a9-82de8e5d8d09 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:57.964730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5405f10-458f-46e0-a9a9-82de8e5d8d09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:57.972418 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5405f10-458f-46e0-a9a9-82de8e5d8d09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:57.974005 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5405f10-458f-46e0-a9a9-82de8e5d8d09 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:57.985398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6226da95-98d0-45d4-aeea-9ce2c2b718c9 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-b5405f10-458f-46e0-a9a9-82de8e5d8d09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:38:57.990382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:57.991578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:57.992770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:38:57.992770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:38:57.992770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:38:57.992770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:38:57.992770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:38:57.992770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:38:57.992770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:38:57.992770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:57.996385 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 498/990] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:38:57.999967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5405f10-458f-46e0-a9a9-82de8e5d8d09 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:58.028353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb29380-ca4c-4200-ac8a-b45a603c2698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:58.029150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb29380-ca4c-4200-ac8a-b45a603c2698 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:58.029641 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb29380-ca4c-4200-ac8a-b45a603c2698 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:58.030001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb29380-ca4c-4200-ac8a-b45a603c2698 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:58.030785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb29380-ca4c-4200-ac8a-b45a603c2698 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:58.031189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb29380-ca4c-4200-ac8a-b45a603c2698 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:58.032876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb29380-ca4c-4200-ac8a-b45a603c2698 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:58.043314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 493/991] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Jul 14 17:38:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:58.044662 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb29380-ca4c-4200-ac8a-b45a603c2698 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:58.078028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:58.123394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5405f10-458f-46e0-a9a9-82de8e5d8d09 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:58.164954 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3796c-e1e6-4e73-849d-fafc00c3f2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:58.164954 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3796c-e1e6-4e73-849d-fafc00c3f2d0 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:58.164954 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3796c-e1e6-4e73-849d-fafc00c3f2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:58.164954 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3796c-e1e6-4e73-849d-fafc00c3f2d0 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:58.165842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3796c-e1e6-4e73-849d-fafc00c3f2d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:58.166162 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3796c-e1e6-4e73-849d-fafc00c3f2d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:58.167596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3796c-e1e6-4e73-849d-fafc00c3f2d0 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:58.169341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb29380-ca4c-4200-ac8a-b45a603c2698 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:58.182808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 499/992] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:38:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:58.189510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3796c-e1e6-4e73-849d-fafc00c3f2d0 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:58.213123 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e885d9-f340-4075-a0a9-1c810a3070e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:58.213864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e885d9-f340-4075-a0a9-1c810a3070e3 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:58.214177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e885d9-f340-4075-a0a9-1c810a3070e3 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:58.214418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e885d9-f340-4075-a0a9-1c810a3070e3 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:58.215624 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e885d9-f340-4075-a0a9-1c810a3070e3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:58.215955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e885d9-f340-4075-a0a9-1c810a3070e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:58.217535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e885d9-f340-4075-a0a9-1c810a3070e3 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:58.219230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:58.223702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 494/993] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Jul 14 17:38:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:58.229588 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e885d9-f340-4075-a0a9-1c810a3070e3 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:58.328423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3796c-e1e6-4e73-849d-fafc00c3f2d0 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:58.347154 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:58.356577 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e885d9-f340-4075-a0a9-1c810a3070e3 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:58.362307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc35f4d-f5d5-49f9-b152-b53cd4046aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:58.362972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc35f4d-f5d5-49f9-b152-b53cd4046aba None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:58.363257 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc35f4d-f5d5-49f9-b152-b53cd4046aba None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:58.363537 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc35f4d-f5d5-49f9-b152-b53cd4046aba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:58.364436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc35f4d-f5d5-49f9-b152-b53cd4046aba None admin] RBAC: Adding query filter params (role.id=086a0517b6e44f8aa33468ec2e3135a7, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=0cd1747822c3493e8aec150a12409fbd) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:58.364728 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc35f4d-f5d5-49f9-b152-b53cd4046aba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:58.366178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc35f4d-f5d5-49f9-b152-b53cd4046aba None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:58.372206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 500/994] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:38:58 2022] GET /identity/v3/role_assignments?user.id=0cd1747822c3493e8aec150a12409fbd&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=086a0517b6e44f8aa33468ec2e3135a7 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:58.374313 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea6b1f-a780-404f-b5a8-8c570cef16b5 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:58.374604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea6b1f-a780-404f-b5a8-8c570cef16b5 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:58.374887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fea6b1f-a780-404f-b5a8-8c570cef16b5 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:58.376193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 501/995] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:38:58.382322 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53a066-5476-4197-97b4-2c352c48b1b0 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:58.382604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53a066-5476-4197-97b4-2c352c48b1b0 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:58.383343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53a066-5476-4197-97b4-2c352c48b1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:58.395501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ed6b1-c539-4e4f-af0d-db7c3cb1f0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:58.396445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ed6b1-c539-4e4f-af0d-db7c3cb1f0a7 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:58.396799 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ed6b1-c539-4e4f-af0d-db7c3cb1f0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:58.397292 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ed6b1-c539-4e4f-af0d-db7c3cb1f0a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:58.398061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ed6b1-c539-4e4f-af0d-db7c3cb1f0a7 None admin] RBAC: Adding query filter params (interface=admin, service_id=f7a1f35ec9b04b9496365925a3b14fa8, region_id=RegionOne) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:58.398453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ed6b1-c539-4e4f-af0d-db7c3cb1f0a7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:58.399966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ed6b1-c539-4e4f-af0d-db7c3cb1f0a7 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:58.401448 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-ca53a066-5476-4197-97b4-2c352c48b1b0 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:38:58.402700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ca53a066-5476-4197-97b4-2c352c48b1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:38:58.407978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 495/996] 10.222.0.64 () {56 vars in 1249 bytes} [Thu Jul 14 17:38:58 2022] GET /identity/v3/endpoints?service_id=f7a1f35ec9b04b9496365925a3b14fa8&interface=admin®ion_id=RegionOne => generated 193 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:58.413158 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ed6b1-c539-4e4f-af0d-db7c3cb1f0a7 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:58.496121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 502/997] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:38:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:38:58.518036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53a066-5476-4197-97b4-2c352c48b1b0 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:58.541774 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ed6b1-c539-4e4f-af0d-db7c3cb1f0a7 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:58.541977 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:58.586254 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c57d19-a856-483b-b12f-f282e123a6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:58.586864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c57d19-a856-483b-b12f-f282e123a6c2 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:58.587292 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c57d19-a856-483b-b12f-f282e123a6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:58.587573 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c57d19-a856-483b-b12f-f282e123a6c2 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:58.588298 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c57d19-a856-483b-b12f-f282e123a6c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:58.589677 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c57d19-a856-483b-b12f-f282e123a6c2 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:58.597628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 496/998] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Jul 14 17:38:58 2022] GET /identity/v3/services => generated 1727 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:38:58.640078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53a066-5476-4197-97b4-2c352c48b1b0 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:58.679775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542a4ff-dd7c-4d19-bad4-5a62361f80ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:58.680788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542a4ff-dd7c-4d19-bad4-5a62361f80ac None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:58.681174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542a4ff-dd7c-4d19-bad4-5a62361f80ac None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:58.681557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542a4ff-dd7c-4d19-bad4-5a62361f80ac None admin] PATH_INFO: `/v3/services/compute` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:58.682342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542a4ff-dd7c-4d19-bad4-5a62361f80ac None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:58.683909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542a4ff-dd7c-4d19-bad4-5a62361f80ac None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-1542a4ff-dd7c-4d19-bad4-5a62361f80ac None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:38:58.699315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:38:58.700443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:38:58.701296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 503/999] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:38:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:38:58.708916 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c57d19-a856-483b-b12f-f282e123a6c2 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:58.736129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:58.824686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c57d19-a856-483b-b12f-f282e123a6c2 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:58.860619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369d8e9-a52d-4e0e-a49d-5b59e61d7284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:58.861214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369d8e9-a52d-4e0e-a49d-5b59e61d7284 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:58.861497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369d8e9-a52d-4e0e-a49d-5b59e61d7284 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:58.861773 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c369d8e9-a52d-4e0e-a49d-5b59e61d7284 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:58.862474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369d8e9-a52d-4e0e-a49d-5b59e61d7284 None admin] RBAC: Adding query filter params (name=compute) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:58.862779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369d8e9-a52d-4e0e-a49d-5b59e61d7284 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:58.864173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c369d8e9-a52d-4e0e-a49d-5b59e61d7284 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:58.869649 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 497/1000] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Jul 14 17:38:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:38:58.876435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542a4ff-dd7c-4d19-bad4-5a62361f80ac None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:58.963380 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:58.983670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542a4ff-dd7c-4d19-bad4-5a62361f80ac None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:59.018397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeec34d-08ee-4430-86f7-f91bd2a693e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:59.019062 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeec34d-08ee-4430-86f7-f91bd2a693e5 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:59.019343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeec34d-08ee-4430-86f7-f91bd2a693e5 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:59.019616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeec34d-08ee-4430-86f7-f91bd2a693e5 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:59.020357 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeec34d-08ee-4430-86f7-f91bd2a693e5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:38:59.020667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeec34d-08ee-4430-86f7-f91bd2a693e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:59.022030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeec34d-08ee-4430-86f7-f91bd2a693e5 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:59.031832 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 504/1001] 10.222.0.64 () {56 vars in 1112 bytes} [Thu Jul 14 17:38:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:38:59.033004 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369d8e9-a52d-4e0e-a49d-5b59e61d7284 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:38:59.131269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:38:59.149122 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c369d8e9-a52d-4e0e-a49d-5b59e61d7284 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:38:59.189022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eaddd1-aaa0-4239-9169-faab9e63be6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:38:59.189691 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eaddd1-aaa0-4239-9169-faab9e63be6a None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:38:59.189971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eaddd1-aaa0-4239-9169-faab9e63be6a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:38:59.190242 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eaddd1-aaa0-4239-9169-faab9e63be6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:38:59.190970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eaddd1-aaa0-4239-9169-faab9e63be6a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:38:59.192539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eaddd1-aaa0-4239-9169-faab9e63be6a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:38:59.219982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 498/1002] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Jul 14 17:38:59 2022] POST /identity/v3/endpoints => generated 334 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:39:00.467204 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda50217-487d-49b1-876f-bf913653cc31 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:00.467742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda50217-487d-49b1-876f-bf913653cc31 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:00.467742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda50217-487d-49b1-876f-bf913653cc31 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:00.469354 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 505/1003] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:00.474966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18e6a1-26c6-42ce-8c77-4e40b8740e16 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:00.475450 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18e6a1-26c6-42ce-8c77-4e40b8740e16 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:00.475527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18e6a1-26c6-42ce-8c77-4e40b8740e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:00.494266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-4e18e6a1-26c6-42ce-8c77-4e40b8740e16 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:00.495449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4e18e6a1-26c6-42ce-8c77-4e40b8740e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:00.595109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 499/1004] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:00 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:00.617935 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda50217-487d-49b1-876f-bf913653cc31 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:00.674408 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b63c5f-7b28-4a83-ac7c-6cfd082abfb0 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:00.675374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b63c5f-7b28-4a83-ac7c-6cfd082abfb0 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:00.676013 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b63c5f-7b28-4a83-ac7c-6cfd082abfb0 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:00.679076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 500/1005] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:39:00.688189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58a896-abfc-4015-a4b8-8aa4596caf3f None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:00.688704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58a896-abfc-4015-a4b8-8aa4596caf3f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:00.689212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58a896-abfc-4015-a4b8-8aa4596caf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:00.717617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-5f58a896-abfc-4015-a4b8-8aa4596caf3f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:00.719589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5f58a896-abfc-4015-a4b8-8aa4596caf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:00.777000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda50217-487d-49b1-876f-bf913653cc31 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:00.794032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:00.809486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 501/1006] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:00 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:00.825709 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e9aef-da18-49a4-9691-0607c2139ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:00.827242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e9aef-da18-49a4-9691-0607c2139ccb None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:00.827557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e9aef-da18-49a4-9691-0607c2139ccb None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:00.827836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e9aef-da18-49a4-9691-0607c2139ccb None admin] PATH_INFO: `/v3/services/image` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:00.828852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e9aef-da18-49a4-9691-0607c2139ccb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:00.831351 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15e9aef-da18-49a4-9691-0607c2139ccb None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:00.833184 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58a896-abfc-4015-a4b8-8aa4596caf3f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-f15e9aef-da18-49a4-9691-0607c2139ccb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:00.844576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:00.845651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:00.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:00.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:00.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:00.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:00.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:00.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:00.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:00.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:00.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:00.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:00.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:39:00.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:39:00.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 17:39:00.846607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:00.850809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 506/1007] 10.222.0.64 () {56 vars in 1105 bytes} [Thu Jul 14 17:39:00 2022] GET /identity/v3/services/image => generated 86 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:00.851574 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e9aef-da18-49a4-9691-0607c2139ccb None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:00.866243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:00.977058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58a896-abfc-4015-a4b8-8aa4596caf3f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:00.985074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e9aef-da18-49a4-9691-0607c2139ccb None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:01.018861 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c00944-385a-482e-87e2-9f690665a2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:01.019882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c00944-385a-482e-87e2-9f690665a2ec None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:01.020196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c00944-385a-482e-87e2-9f690665a2ec None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:01.020536 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c00944-385a-482e-87e2-9f690665a2ec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:01.021153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c00944-385a-482e-87e2-9f690665a2ec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:01.022595 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c00944-385a-482e-87e2-9f690665a2ec None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:01.023745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5b825-bcce-43d9-b06e-05e2e7b35265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:01.024448 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5b825-bcce-43d9-b06e-05e2e7b35265 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:01.024940 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5b825-bcce-43d9-b06e-05e2e7b35265 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:01.025222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5b825-bcce-43d9-b06e-05e2e7b35265 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:01.026299 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5b825-bcce-43d9-b06e-05e2e7b35265 None admin] RBAC: Adding query filter params (name=image) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:01.026772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5b825-bcce-43d9-b06e-05e2e7b35265 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:01.028318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5b825-bcce-43d9-b06e-05e2e7b35265 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:01.034383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 507/1008] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-b0c00944-385a-482e-87e2-9f690665a2ec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:01.037017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:01.038134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:01.045088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 502/1009] 10.222.0.64 () {56 vars in 1126 bytes} [Thu Jul 14 17:39:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:39:01.046151 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5b825-bcce-43d9-b06e-05e2e7b35265 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:01.046251 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c00944-385a-482e-87e2-9f690665a2ec None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:01.124886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:01.151036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:01.177613 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c00944-385a-482e-87e2-9f690665a2ec None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:01.191857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5b825-bcce-43d9-b06e-05e2e7b35265 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:01.220028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4a07f-9af3-4182-9ce4-e67f94edcf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:01.220727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4a07f-9af3-4182-9ce4-e67f94edcf79 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:01.221010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4a07f-9af3-4182-9ce4-e67f94edcf79 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:01.221391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4a07f-9af3-4182-9ce4-e67f94edcf79 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:01.222251 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4a07f-9af3-4182-9ce4-e67f94edcf79 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:01.222643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4a07f-9af3-4182-9ce4-e67f94edcf79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:01.224307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4a07f-9af3-4182-9ce4-e67f94edcf79 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:01.235333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 503/1010] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Jul 14 17:39:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:01.237110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4a07f-9af3-4182-9ce4-e67f94edcf79 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:01.239190 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fda90b-91b6-4854-8e8d-8ad386effe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:01.239970 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fda90b-91b6-4854-8e8d-8ad386effe07 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:01.240268 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fda90b-91b6-4854-8e8d-8ad386effe07 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:01.240554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fda90b-91b6-4854-8e8d-8ad386effe07 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:01.241327 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fda90b-91b6-4854-8e8d-8ad386effe07 None admin] RBAC: Adding query filter params (type=image) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:01.241645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fda90b-91b6-4854-8e8d-8ad386effe07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:01.243131 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fda90b-91b6-4854-8e8d-8ad386effe07 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:01.260151 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 508/1011] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:01.395752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4a07f-9af3-4182-9ce4-e67f94edcf79 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:01.401143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4b66b-d954-43b6-a02b-dcb790d4568b None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:01.401553 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4b66b-d954-43b6-a02b-dcb790d4568b None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:01.401736 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e4b66b-d954-43b6-a02b-dcb790d4568b None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:01.403260 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 509/1012] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:01.409275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c80e0-3f8e-4f35-ad5b-e8af79550874 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:01.409578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c80e0-3f8e-4f35-ad5b-e8af79550874 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:01.409851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c80e0-3f8e-4f35-ad5b-e8af79550874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:01.428216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-975c80e0-3f8e-4f35-ad5b-e8af79550874 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:01.429260 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-975c80e0-3f8e-4f35-ad5b-e8af79550874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:01.440176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:01.460963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005a18f-28dc-442a-bdff-af85f526d21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:01.462042 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005a18f-28dc-442a-bdff-af85f526d21c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:01.462485 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005a18f-28dc-442a-bdff-af85f526d21c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:01.462918 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005a18f-28dc-442a-bdff-af85f526d21c None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:01.464052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005a18f-28dc-442a-bdff-af85f526d21c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:01.464505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005a18f-28dc-442a-bdff-af85f526d21c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:01.466641 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005a18f-28dc-442a-bdff-af85f526d21c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:01.475293 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 504/1013] 10.222.0.64 () {56 vars in 1122 bytes} [Thu Jul 14 17:39:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:01.481120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005a18f-28dc-442a-bdff-af85f526d21c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:01.528464 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 510/1014] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:01 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:39:01.568733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c80e0-3f8e-4f35-ad5b-e8af79550874 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:01.622920 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005a18f-28dc-442a-bdff-af85f526d21c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:01.672550 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4751bc5-37b5-4e04-8175-53e9f236b8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:01.674323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4751bc5-37b5-4e04-8175-53e9f236b8b9 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:01.674824 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4751bc5-37b5-4e04-8175-53e9f236b8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:01.675301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4751bc5-37b5-4e04-8175-53e9f236b8b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:01.676934 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4751bc5-37b5-4e04-8175-53e9f236b8b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:01.679528 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4751bc5-37b5-4e04-8175-53e9f236b8b9 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:01.682794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:01.709511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 505/1015] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Jul 14 17:39:01 2022] POST /identity/v3/endpoints => generated 325 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:39:01.741510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c80e0-3f8e-4f35-ad5b-e8af79550874 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:01.826741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1b1e6-28ec-4e7b-b7ff-221cc009e0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:01.828081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1b1e6-28ec-4e7b-b7ff-221cc009e0c0 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:01.828571 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1b1e6-28ec-4e7b-b7ff-221cc009e0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:01.829041 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1b1e6-28ec-4e7b-b7ff-221cc009e0c0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:01.849568 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1b1e6-28ec-4e7b-b7ff-221cc009e0c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-2dd1b1e6-28ec-4e7b-b7ff-221cc009e0c0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:39:01.866474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:01.868343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:01.869936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:01.869936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:01.869936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:01.869936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:01.869936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:39:01.869936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:39:01.869936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:39:01.869936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:39:01.869936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:39:01.869936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:39:01.869936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 17:39:01.869936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:01.872199 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 511/1016] 10.222.0.64 () {56 vars in 1120 bytes} [Thu Jul 14 17:39:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:39:01.880169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4751bc5-37b5-4e04-8175-53e9f236b8b9 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:01.957523 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:02.056746 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4751bc5-37b5-4e04-8175-53e9f236b8b9 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:02.124329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284f2dd-18fc-4639-bc23-6ade764d46b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:02.124329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284f2dd-18fc-4639-bc23-6ade764d46b6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:02.124329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284f2dd-18fc-4639-bc23-6ade764d46b6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:02.124329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284f2dd-18fc-4639-bc23-6ade764d46b6 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:02.125223 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284f2dd-18fc-4639-bc23-6ade764d46b6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:02.125223 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284f2dd-18fc-4639-bc23-6ade764d46b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:02.126988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284f2dd-18fc-4639-bc23-6ade764d46b6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:02.148007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 506/1017] 10.222.0.64 () {56 vars in 1115 bytes} [Thu Jul 14 17:39:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:02.152451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1b1e6-28ec-4e7b-b7ff-221cc009e0c0 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:02.269696 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1b1e6-28ec-4e7b-b7ff-221cc009e0c0 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:02.297265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:02.311603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f0295-e6fe-4c59-9f5f-6675fa9fba05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:02.312629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f0295-e6fe-4c59-9f5f-6675fa9fba05 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:02.313042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f0295-e6fe-4c59-9f5f-6675fa9fba05 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:02.313465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f0295-e6fe-4c59-9f5f-6675fa9fba05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:02.325969 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f0295-e6fe-4c59-9f5f-6675fa9fba05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:02.327786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f0295-e6fe-4c59-9f5f-6675fa9fba05 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-e27f0295-e6fe-4c59-9f5f-6675fa9fba05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:39:02.348466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:02.350246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:02.356540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:39:02.356540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:39:02.356540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:39:02.356540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:02.356540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 512/1018] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:02.359268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284f2dd-18fc-4639-bc23-6ade764d46b6 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:02.477060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284f2dd-18fc-4639-bc23-6ade764d46b6 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:02.489558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:02.518700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b2c4d-ecad-4c7d-a1a5-d8862d3d6fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:02.519571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b2c4d-ecad-4c7d-a1a5-d8862d3d6fce None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:02.520073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b2c4d-ecad-4c7d-a1a5-d8862d3d6fce None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:02.520501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b2c4d-ecad-4c7d-a1a5-d8862d3d6fce None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:02.521395 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b2c4d-ecad-4c7d-a1a5-d8862d3d6fce None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:02.521833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b2c4d-ecad-4c7d-a1a5-d8862d3d6fce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:02.523345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b2c4d-ecad-4c7d-a1a5-d8862d3d6fce None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:02.538053 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 507/1019] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:02.544360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f0295-e6fe-4c59-9f5f-6675fa9fba05 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:02.656634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f0295-e6fe-4c59-9f5f-6675fa9fba05 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:02.693308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333fb5f-60a9-45cf-a6fd-5a413cd7b0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:02.694364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333fb5f-60a9-45cf-a6fd-5a413cd7b0a1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:02.694828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333fb5f-60a9-45cf-a6fd-5a413cd7b0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:02.695254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a333fb5f-60a9-45cf-a6fd-5a413cd7b0a1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:02.704188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333fb5f-60a9-45cf-a6fd-5a413cd7b0a1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:02.705870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a333fb5f-60a9-45cf-a6fd-5a413cd7b0a1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-a333fb5f-60a9-45cf-a6fd-5a413cd7b0a1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:39:02.721139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:02.722287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:02.723258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:02.723258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:02.723258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:39:02.723258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:39:02.723258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:39:02.723258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:39:02.723258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:39:02.723258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:39:02.723258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:39:02.723258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:02.729046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 513/1020] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Jul 14 17:39:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:39:02.732281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b2c4d-ecad-4c7d-a1a5-d8862d3d6fce None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:02.847836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b2c4d-ecad-4c7d-a1a5-d8862d3d6fce None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:02.890058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387b806-221d-4767-8cdd-2d1d65f02c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:02.891041 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387b806-221d-4767-8cdd-2d1d65f02c67 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:02.891481 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387b806-221d-4767-8cdd-2d1d65f02c67 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:02.891908 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387b806-221d-4767-8cdd-2d1d65f02c67 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:02.892928 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387b806-221d-4767-8cdd-2d1d65f02c67 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:02.893404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387b806-221d-4767-8cdd-2d1d65f02c67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:02.895002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387b806-221d-4767-8cdd-2d1d65f02c67 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:02.919993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 508/1021] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Jul 14 17:39:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:02.922814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333fb5f-60a9-45cf-a6fd-5a413cd7b0a1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:02.973508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:03.049311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a333fb5f-60a9-45cf-a6fd-5a413cd7b0a1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:03.088181 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136269e5-92ae-401d-8a79-06f365cc0ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:03.088835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136269e5-92ae-401d-8a79-06f365cc0ece None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:03.089112 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136269e5-92ae-401d-8a79-06f365cc0ece None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:03.089387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136269e5-92ae-401d-8a79-06f365cc0ece None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:03.175745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136269e5-92ae-401d-8a79-06f365cc0ece None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:03.177296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136269e5-92ae-401d-8a79-06f365cc0ece None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:03.216753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:03.303277 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 514/1022] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:39:02 2022] GET /identity/v3/auth/tokens => generated 1863 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:39:03.304672 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387b806-221d-4767-8cdd-2d1d65f02c67 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:03.337154 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:03.422604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387b806-221d-4767-8cdd-2d1d65f02c67 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:03.481648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da672b-9848-4fa0-9cfb-8ee755c07693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:03.483564 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da672b-9848-4fa0-9cfb-8ee755c07693 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:03.484097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da672b-9848-4fa0-9cfb-8ee755c07693 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:03.484590 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da672b-9848-4fa0-9cfb-8ee755c07693 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:03.504785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da672b-9848-4fa0-9cfb-8ee755c07693 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:03.507500 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da672b-9848-4fa0-9cfb-8ee755c07693 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-a5da672b-9848-4fa0-9cfb-8ee755c07693 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:39:03.540521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:03.541487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:03.542579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:39:03.542579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:39:03.542579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:39:03.542579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:03.542948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 509/1023] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:39:03.552634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136269e5-92ae-401d-8a79-06f365cc0ece None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:03.658984 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cf21f-c5a1-4176-b843-7437e4656cd5 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:03.659575 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cf21f-c5a1-4176-b843-7437e4656cd5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:03.660233 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5cf21f-c5a1-4176-b843-7437e4656cd5 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:03.662943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 510/1024] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:39:03.671740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff8566-4248-4c3a-a8fd-ce857aaadcd0 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:03.672316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff8566-4248-4c3a-a8fd-ce857aaadcd0 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:03.672984 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff8566-4248-4c3a-a8fd-ce857aaadcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:03.701606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-41ff8566-4248-4c3a-a8fd-ce857aaadcd0 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:03.704439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-41ff8566-4248-4c3a-a8fd-ce857aaadcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:03.738749 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136269e5-92ae-401d-8a79-06f365cc0ece None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:03.810855 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130191fe-4868-41e1-902c-bc133de753cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:03.811647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 511/1025] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:03 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:39:03.813024 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130191fe-4868-41e1-902c-bc133de753cc None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:03.813797 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130191fe-4868-41e1-902c-bc133de753cc None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:03.814498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130191fe-4868-41e1-902c-bc133de753cc None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:03.816107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130191fe-4868-41e1-902c-bc133de753cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:03.816868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130191fe-4868-41e1-902c-bc133de753cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:03.819744 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130191fe-4868-41e1-902c-bc133de753cc None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:03.824591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:03.843764 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff8566-4248-4c3a-a8fd-ce857aaadcd0 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:03.851578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 515/1026] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:03.853178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130191fe-4868-41e1-902c-bc133de753cc None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:03.869499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:04.023478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff8566-4248-4c3a-a8fd-ce857aaadcd0 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:04.025138 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130191fe-4868-41e1-902c-bc133de753cc None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:04.076188 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272c8ca-dcab-48c6-bd75-1064fae84709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:04.077826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272c8ca-dcab-48c6-bd75-1064fae84709 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:04.078483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272c8ca-dcab-48c6-bd75-1064fae84709 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:04.078976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2272c8ca-dcab-48c6-bd75-1064fae84709 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:04.080393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272c8ca-dcab-48c6-bd75-1064fae84709 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:04.083034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2272c8ca-dcab-48c6-bd75-1064fae84709 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:04.085262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cebac-6abf-41aa-9f99-280d714594a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:04.086201 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cebac-6abf-41aa-9f99-280d714594a1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:04.086615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cebac-6abf-41aa-9f99-280d714594a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:04.087060 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40cebac-6abf-41aa-9f99-280d714594a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:04.097235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40cebac-6abf-41aa-9f99-280d714594a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:04.100074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40cebac-6abf-41aa-9f99-280d714594a1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:04.112208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 512/1027] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Jul 14 17:39:03 2022] POST /identity/v3/services => generated 242 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-b40cebac-6abf-41aa-9f99-280d714594a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:39:04.123561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:04.126819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:04.129189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:39:04.129189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:39:04.129189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:39:04.129189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:39:04.129189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:39:04.129189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:39:04.129189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:39:04.129189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:04.134088 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 516/1028] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:39:04.136153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272c8ca-dcab-48c6-bd75-1064fae84709 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:04.234097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:04.286063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2272c8ca-dcab-48c6-bd75-1064fae84709 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:04.328743 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd306a-c456-4d2e-b371-e0548dabd811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:04.329667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd306a-c456-4d2e-b371-e0548dabd811 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:04.330096 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd306a-c456-4d2e-b371-e0548dabd811 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:04.330508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd306a-c456-4d2e-b371-e0548dabd811 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:04.331473 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd306a-c456-4d2e-b371-e0548dabd811 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:04.331919 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd306a-c456-4d2e-b371-e0548dabd811 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:04.333586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd306a-c456-4d2e-b371-e0548dabd811 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:04.350346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 513/1029] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:39:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:04.359899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cebac-6abf-41aa-9f99-280d714594a1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:04.443420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:04.526617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40cebac-6abf-41aa-9f99-280d714594a1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:04.593749 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf6f63d-5abc-48bf-8b86-4a5ca6e6c213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:04.594860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6f63d-5abc-48bf-8b86-4a5ca6e6c213 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:04.595362 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6f63d-5abc-48bf-8b86-4a5ca6e6c213 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:04.595916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf6f63d-5abc-48bf-8b86-4a5ca6e6c213 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:04.597622 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6f63d-5abc-48bf-8b86-4a5ca6e6c213 None admin] RBAC: Adding query filter params (role.id=086a0517b6e44f8aa33468ec2e3135a7, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=6423f54152064b909d03c0908d636993) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:04.598200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6f63d-5abc-48bf-8b86-4a5ca6e6c213 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:04.601073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf6f63d-5abc-48bf-8b86-4a5ca6e6c213 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:04.616048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 517/1030] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:39:04 2022] GET /identity/v3/role_assignments?user.id=6423f54152064b909d03c0908d636993&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=086a0517b6e44f8aa33468ec2e3135a7 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:04.873351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad55bb8-627f-4ba4-897f-61272a3d2408 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:04.874937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad55bb8-627f-4ba4-897f-61272a3d2408 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:04.875088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad55bb8-627f-4ba4-897f-61272a3d2408 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:04.877789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 514/1031] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:39:04.886620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32632fd7-9f95-4bf1-81c3-99f826a01a68 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:04.887119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32632fd7-9f95-4bf1-81c3-99f826a01a68 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:04.887613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32632fd7-9f95-4bf1-81c3-99f826a01a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:04.918462 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-32632fd7-9f95-4bf1-81c3-99f826a01a68 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:04.921071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-32632fd7-9f95-4bf1-81c3-99f826a01a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:05.056874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 518/1032] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:05.098986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad55bb8-627f-4ba4-897f-61272a3d2408 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:05.305781 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad55bb8-627f-4ba4-897f-61272a3d2408 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:05.360512 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:05.377803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c527e8d-6910-4a43-acf6-d91a9b383ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:05.379103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c527e8d-6910-4a43-acf6-d91a9b383ed4 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:05.379542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c527e8d-6910-4a43-acf6-d91a9b383ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:05.380047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c527e8d-6910-4a43-acf6-d91a9b383ed4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:05.382200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c527e8d-6910-4a43-acf6-d91a9b383ed4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:05.384834 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c527e8d-6910-4a43-acf6-d91a9b383ed4 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:05.388098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 515/1033] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:05.397845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32632fd7-9f95-4bf1-81c3-99f826a01a68 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:05.493295 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:05.590526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32632fd7-9f95-4bf1-81c3-99f826a01a68 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:05.656271 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cf840-5227-41e7-909d-d64ea36ffea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:05.657389 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cf840-5227-41e7-909d-d64ea36ffea9 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:05.657889 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cf840-5227-41e7-909d-d64ea36ffea9 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:05.658383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cf840-5227-41e7-909d-d64ea36ffea9 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:05.659470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79cf840-5227-41e7-909d-d64ea36ffea9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:05.662109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79cf840-5227-41e7-909d-d64ea36ffea9 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:05.708994 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 519/1034] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Jul 14 17:39:05 2022] POST /identity/v3/projects => generated 311 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:39:06.424056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8435bde-bebc-4722-bc90-04192fd9295d None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:06.424554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8435bde-bebc-4722-bc90-04192fd9295d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:06.424554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8435bde-bebc-4722-bc90-04192fd9295d None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:06.426118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 516/1035] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:06.431335 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1f433-98cf-4a8d-bc5a-083bc7aa21b8 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:06.431626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1f433-98cf-4a8d-bc5a-083bc7aa21b8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:06.431905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf1f433-98cf-4a8d-bc5a-083bc7aa21b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:06.449968 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-aaf1f433-98cf-4a8d-bc5a-083bc7aa21b8 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:06.451169 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-aaf1f433-98cf-4a8d-bc5a-083bc7aa21b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:06.526179 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 520/1036] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:06.552253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8435bde-bebc-4722-bc90-04192fd9295d None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:06.669974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8435bde-bebc-4722-bc90-04192fd9295d None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:06.709414 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f0bf7-896f-4f38-b8db-77a21e334fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:06.710221 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f0bf7-896f-4f38-b8db-77a21e334fcc None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:06.710719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f0bf7-896f-4f38-b8db-77a21e334fcc None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:06.710870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f0bf7-896f-4f38-b8db-77a21e334fcc None admin] PATH_INFO: `/v3/services/image` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:06.711428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f0bf7-896f-4f38-b8db-77a21e334fcc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:06.712847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f0bf7-896f-4f38-b8db-77a21e334fcc None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-c34f0bf7-896f-4f38-b8db-77a21e334fcc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:06.725819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:06.726922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:06.727829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:06.727829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:06.727829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:06.727829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:06.727829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:06.727829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:06.727829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:06.727829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:06.727829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:06.727829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:06.727829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:39:06.727829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:39:06.727829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 17:39:06.727829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:06.732243 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 517/1037] 10.222.0.64 () {56 vars in 1105 bytes} [Thu Jul 14 17:39:06 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:06.733492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1f433-98cf-4a8d-bc5a-083bc7aa21b8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:06.770237 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:06.845932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf1f433-98cf-4a8d-bc5a-083bc7aa21b8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:06.881383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce08b3a-c31e-437b-92eb-fca243500fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:06.882052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce08b3a-c31e-437b-92eb-fca243500fa1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:06.882335 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce08b3a-c31e-437b-92eb-fca243500fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:06.882611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce08b3a-c31e-437b-92eb-fca243500fa1 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:06.883384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce08b3a-c31e-437b-92eb-fca243500fa1 None admin] RBAC: Adding query filter params (name=image) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:06.883697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce08b3a-c31e-437b-92eb-fca243500fa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:06.885149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce08b3a-c31e-437b-92eb-fca243500fa1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:06.895350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 521/1038] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:06.896952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f0bf7-896f-4f38-b8db-77a21e334fcc None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:06.958534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0578fa-66ea-40ae-83e9-866ce773ce0d None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:06.958865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0578fa-66ea-40ae-83e9-866ce773ce0d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:06.959100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0578fa-66ea-40ae-83e9-866ce773ce0d None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:06.959881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:06.960778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 522/1039] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:06.966123 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37fbe3-da07-4be0-9c04-446596f652b0 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:06.966416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37fbe3-da07-4be0-9c04-446596f652b0 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:06.966695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e37fbe3-da07-4be0-9c04-446596f652b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:06.983905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-3e37fbe3-da07-4be0-9c04-446596f652b0 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:06.985136 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3e37fbe3-da07-4be0-9c04-446596f652b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:07.006905 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f0bf7-896f-4f38-b8db-77a21e334fcc None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:07.044659 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d4b3a-cca3-414a-81e3-c1a4bb6a5609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:07.045329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d4b3a-cca3-414a-81e3-c1a4bb6a5609 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:07.046138 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d4b3a-cca3-414a-81e3-c1a4bb6a5609 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:07.046420 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d4b3a-cca3-414a-81e3-c1a4bb6a5609 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:07.047305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d4b3a-cca3-414a-81e3-c1a4bb6a5609 None admin] RBAC: Adding query filter params (type=image) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:07.047765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d4b3a-cca3-414a-81e3-c1a4bb6a5609 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:07.049163 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d4b3a-cca3-414a-81e3-c1a4bb6a5609 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:07.058413 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 523/1040] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:39:07.060124 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 518/1041] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:07.060536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37fbe3-da07-4be0-9c04-446596f652b0 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:07.082408 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d4b3a-cca3-414a-81e3-c1a4bb6a5609 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:07.089110 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:07.177674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e37fbe3-da07-4be0-9c04-446596f652b0 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:07.198165 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d4b3a-cca3-414a-81e3-c1a4bb6a5609 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:07.217063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cd6ba-526e-438c-bd28-f21f4ab28976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:07.218236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cd6ba-526e-438c-bd28-f21f4ab28976 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:07.218523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cd6ba-526e-438c-bd28-f21f4ab28976 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:07.218800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cd6ba-526e-438c-bd28-f21f4ab28976 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:07.219598 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50cd6ba-526e-438c-bd28-f21f4ab28976 None admin] RBAC: Adding query filter params (interface=public, service_id=9579438f677f4c1ab0c02b8f4a270f35, region_id=RegionOne) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:07.220348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50cd6ba-526e-438c-bd28-f21f4ab28976 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:07.222051 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50cd6ba-526e-438c-bd28-f21f4ab28976 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:07.222781 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:07.230397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 524/1042] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Jul 14 17:39:07 2022] GET /identity/v3/endpoints?service_id=9579438f677f4c1ab0c02b8f4a270f35&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:07.236091 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cd6ba-526e-438c-bd28-f21f4ab28976 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:07.238403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109b12c-2dea-41ba-801a-5b1c6ff930ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:07.239539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109b12c-2dea-41ba-801a-5b1c6ff930ef None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:07.240739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109b12c-2dea-41ba-801a-5b1c6ff930ef None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:07.241028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109b12c-2dea-41ba-801a-5b1c6ff930ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:07.253876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109b12c-2dea-41ba-801a-5b1c6ff930ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-a109b12c-2dea-41ba-801a-5b1c6ff930ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:39:07.262194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:07.263170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:07.264120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:07.264120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:07.264120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:07.264120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:07.264120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:39:07.264120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:39:07.264120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:39:07.264120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:39:07.264120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:39:07.264120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:39:07.264120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 17:39:07.264120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:07.264120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 519/1043] 10.222.0.64 () {56 vars in 1120 bytes} [Thu Jul 14 17:39:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:39:07.269981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109b12c-2dea-41ba-801a-5b1c6ff930ef None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:07.335791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:07.346913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cd6ba-526e-438c-bd28-f21f4ab28976 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:07.384531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109b12c-2dea-41ba-801a-5b1c6ff930ef None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:07.385528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd493ce-ff71-4d5b-8aee-ec95823bd439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:07.386365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd493ce-ff71-4d5b-8aee-ec95823bd439 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:07.386652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd493ce-ff71-4d5b-8aee-ec95823bd439 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:07.387746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd493ce-ff71-4d5b-8aee-ec95823bd439 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:07.388475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd493ce-ff71-4d5b-8aee-ec95823bd439 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:07.389886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd493ce-ff71-4d5b-8aee-ec95823bd439 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:07.408292 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 525/1044] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Jul 14 17:39:07 2022] GET /identity/v3/services => generated 1957 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:39:07.422469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49882e-9989-4108-86b4-81b46317143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:07.423177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49882e-9989-4108-86b4-81b46317143a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:07.423464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49882e-9989-4108-86b4-81b46317143a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:07.423747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49882e-9989-4108-86b4-81b46317143a None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:07.424570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49882e-9989-4108-86b4-81b46317143a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:07.425133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49882e-9989-4108-86b4-81b46317143a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:07.426303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db49882e-9989-4108-86b4-81b46317143a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:07.441318 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 520/1045] 10.222.0.64 () {56 vars in 1115 bytes} [Thu Jul 14 17:39:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:07.442557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd493ce-ff71-4d5b-8aee-ec95823bd439 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:07.557140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd493ce-ff71-4d5b-8aee-ec95823bd439 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:07.594402 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a777d6c-2a0f-4b75-ad7c-3ea7cc5cd041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:07.595213 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a777d6c-2a0f-4b75-ad7c-3ea7cc5cd041 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:07.595495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a777d6c-2a0f-4b75-ad7c-3ea7cc5cd041 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:07.595772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a777d6c-2a0f-4b75-ad7c-3ea7cc5cd041 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:07.606928 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a777d6c-2a0f-4b75-ad7c-3ea7cc5cd041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:07.609052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a777d6c-2a0f-4b75-ad7c-3ea7cc5cd041 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-2a777d6c-2a0f-4b75-ad7c-3ea7cc5cd041 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:39:07.627752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:07.628793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:07.629717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:39:07.629717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:39:07.629717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:39:07.629717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:07.629717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 526/1046] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:07.635460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49882e-9989-4108-86b4-81b46317143a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:07.688806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:07.761871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db49882e-9989-4108-86b4-81b46317143a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:07.797336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b7c4e-f26d-4b13-af7c-dbfa525cf973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:07.798007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7c4e-f26d-4b13-af7c-dbfa525cf973 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:07.798550 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7c4e-f26d-4b13-af7c-dbfa525cf973 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:07.798550 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7c4e-f26d-4b13-af7c-dbfa525cf973 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:07.799295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b7c4e-f26d-4b13-af7c-dbfa525cf973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:07.799604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b7c4e-f26d-4b13-af7c-dbfa525cf973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:07.801006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b7c4e-f26d-4b13-af7c-dbfa525cf973 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:07.818919 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 521/1047] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:39:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:07.820692 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a777d6c-2a0f-4b75-ad7c-3ea7cc5cd041 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:07.831617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:07.934083 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a777d6c-2a0f-4b75-ad7c-3ea7cc5cd041 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:07.971879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3ed64-187e-4b60-875a-54ee574589d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:07.972731 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3ed64-187e-4b60-875a-54ee574589d1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:07.973013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3ed64-187e-4b60-875a-54ee574589d1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:07.973318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde3ed64-187e-4b60-875a-54ee574589d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:07.976785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcef016-373b-4787-9a38-de36b8c8d394 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:07.977078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcef016-373b-4787-9a38-de36b8c8d394 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:07.977358 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcef016-373b-4787-9a38-de36b8c8d394 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:07.978755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 522/1048] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:39:07.982618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3ed64-187e-4b60-875a-54ee574589d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:07.984712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde3ed64-187e-4b60-875a-54ee574589d1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:07.984829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2e00-ca97-4631-bfb3-253efd687316 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:07.984998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2e00-ca97-4631-bfb3-253efd687316 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:07.985264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b2e00-ca97-4631-bfb3-253efd687316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-cde3ed64-187e-4b60-875a-54ee574589d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:39:08.001383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:08.002615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:08.003646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:08.003646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:08.003646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:39:08.003646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:39:08.003646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:39:08.003646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:39:08.003646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:39:08.003646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:39:08.003646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:39:08.003646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:08.003646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 527/1049] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Jul 14 17:39:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:08.005889 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-474b2e00-ca97-4631-bfb3-253efd687316 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:08.007114 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-474b2e00-ca97-4631-bfb3-253efd687316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:08.008580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3ed64-187e-4b60-875a-54ee574589d1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:08.081216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 523/1050] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:08.083675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:08.111187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b2e00-ca97-4631-bfb3-253efd687316 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:08.129448 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde3ed64-187e-4b60-875a-54ee574589d1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:08.154847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:08.173584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de32cf-3bf4-4fad-a2d8-3dc204e12295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:08.174660 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de32cf-3bf4-4fad-a2d8-3dc204e12295 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:08.175125 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de32cf-3bf4-4fad-a2d8-3dc204e12295 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:08.175626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de32cf-3bf4-4fad-a2d8-3dc204e12295 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:08.176951 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de32cf-3bf4-4fad-a2d8-3dc204e12295 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:08.177302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de32cf-3bf4-4fad-a2d8-3dc204e12295 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:08.179654 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de32cf-3bf4-4fad-a2d8-3dc204e12295 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:08.204392 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 528/1051] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Jul 14 17:39:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:08.210181 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de32cf-3bf4-4fad-a2d8-3dc204e12295 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:08.257732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474b2e00-ca97-4631-bfb3-253efd687316 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:08.299993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c0ba9-ad26-450f-9db7-970b5eb91233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:08.301579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c0ba9-ad26-450f-9db7-970b5eb91233 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:08.301775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c0ba9-ad26-450f-9db7-970b5eb91233 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:08.302262 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c0ba9-ad26-450f-9db7-970b5eb91233 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:08.303615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c0ba9-ad26-450f-9db7-970b5eb91233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:08.305496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c0ba9-ad26-450f-9db7-970b5eb91233 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:08.307390 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 524/1052] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:08.313658 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c0ba9-ad26-450f-9db7-970b5eb91233 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:08.330478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de32cf-3bf4-4fad-a2d8-3dc204e12295 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:08.359254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:08.373366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75692466-0873-492e-8448-49891292da2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:08.374957 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75692466-0873-492e-8448-49891292da2f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:08.375240 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75692466-0873-492e-8448-49891292da2f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:08.375505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75692466-0873-492e-8448-49891292da2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:08.387655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75692466-0873-492e-8448-49891292da2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:08.389287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75692466-0873-492e-8448-49891292da2f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-75692466-0873-492e-8448-49891292da2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:39:08.411158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:08.412383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:08.413298 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:39:08.413298 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:39:08.413298 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:39:08.413298 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:08.417731 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 529/1053] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:39:08.418667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75692466-0873-492e-8448-49891292da2f None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:08.432750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c0ba9-ad26-450f-9db7-970b5eb91233 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:08.434522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:08.477190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e829de4-37de-42eb-9229-9dbd6a1be409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:08.477876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e829de4-37de-42eb-9229-9dbd6a1be409 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:08.478166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e829de4-37de-42eb-9229-9dbd6a1be409 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:08.478446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e829de4-37de-42eb-9229-9dbd6a1be409 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:08.479177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e829de4-37de-42eb-9229-9dbd6a1be409 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:08.480707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e829de4-37de-42eb-9229-9dbd6a1be409 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:08.511959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 525/1054] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:39:08 2022] POST /identity/v3/users => generated 285 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:39:08.550173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75692466-0873-492e-8448-49891292da2f None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:08.595065 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac28ebf-4311-4c51-9bf5-777dd2fee23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:08.595765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac28ebf-4311-4c51-9bf5-777dd2fee23b None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:08.596089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac28ebf-4311-4c51-9bf5-777dd2fee23b None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:08.596371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac28ebf-4311-4c51-9bf5-777dd2fee23b None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:08.597134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac28ebf-4311-4c51-9bf5-777dd2fee23b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:08.597448 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac28ebf-4311-4c51-9bf5-777dd2fee23b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:08.598891 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac28ebf-4311-4c51-9bf5-777dd2fee23b None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:08.614338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 530/1055] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:39:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:08.621290 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e829de4-37de-42eb-9229-9dbd6a1be409 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:08.789305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e829de4-37de-42eb-9229-9dbd6a1be409 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:08.830198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e897ab-163d-4775-b689-296d0bfc1138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:08.831257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e897ab-163d-4775-b689-296d0bfc1138 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:08.831541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e897ab-163d-4775-b689-296d0bfc1138 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:08.831815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e897ab-163d-4775-b689-296d0bfc1138 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:08.840314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e897ab-163d-4775-b689-296d0bfc1138 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:08.841808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e897ab-163d-4775-b689-296d0bfc1138 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-50e897ab-163d-4775-b689-296d0bfc1138 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:39:08.857300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:08.858904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:08.859855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:39:08.859855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:39:08.859855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:39:08.859855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:39:08.859855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:39:08.859855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:39:08.859855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:39:08.859855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:08.859855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 526/1056] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:39:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:08.866926 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac28ebf-4311-4c51-9bf5-777dd2fee23b None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:08.926372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:09.028874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac28ebf-4311-4c51-9bf5-777dd2fee23b None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:09.065309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef24661-e76c-4cc6-a0fd-226f77d32785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:09.065976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef24661-e76c-4cc6-a0fd-226f77d32785 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:09.066265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef24661-e76c-4cc6-a0fd-226f77d32785 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:09.066516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef24661-e76c-4cc6-a0fd-226f77d32785 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:09.067311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef24661-e76c-4cc6-a0fd-226f77d32785 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:09.067646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef24661-e76c-4cc6-a0fd-226f77d32785 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:09.069073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef24661-e76c-4cc6-a0fd-226f77d32785 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:09.084570 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 531/1057] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:39:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:09.089970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e897ab-163d-4775-b689-296d0bfc1138 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:09.110614 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:09.207822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e897ab-163d-4775-b689-296d0bfc1138 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:09.246520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb0efd-03c2-40b3-96c7-d8fe11a39094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:09.247443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb0efd-03c2-40b3-96c7-d8fe11a39094 None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:09.247722 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb0efd-03c2-40b3-96c7-d8fe11a39094 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:09.248034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb0efd-03c2-40b3-96c7-d8fe11a39094 None admin] PATH_INFO: `/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/6423f54152064b909d03c0908d636993/roles/086a0517b6e44f8aa33468ec2e3135a7` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:09.251305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb0efd-03c2-40b3-96c7-d8fe11a39094 None admin] RBAC: Authorizing `identity:create_grant(project_id=de62d2fb17f846f2bce5bef108353fd8, user_id=6423f54152064b909d03c0908d636993, role_id=086a0517b6e44f8aa33468ec2e3135a7)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:09.252985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb0efd-03c2-40b3-96c7-d8fe11a39094 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:09.274627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 527/1058] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:39:09 2022] PUT /identity/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/6423f54152064b909d03c0908d636993/roles/086a0517b6e44f8aa33468ec2e3135a7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:39:09.572533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7170fab-51e9-439c-9d82-4733e341b875 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:09.572818 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7170fab-51e9-439c-9d82-4733e341b875 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:09.573103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7170fab-51e9-439c-9d82-4733e341b875 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:09.574493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 532/1059] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:39:09.579838 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d998caa-2482-4fc8-9808-be94bd352fd6 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:09.580361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d998caa-2482-4fc8-9808-be94bd352fd6 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:09.580458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d998caa-2482-4fc8-9808-be94bd352fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:09.597246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-9d998caa-2482-4fc8-9808-be94bd352fd6 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:09.598428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9d998caa-2482-4fc8-9808-be94bd352fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:09.667079 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 528/1060] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:09.690374 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7170fab-51e9-439c-9d82-4733e341b875 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:09.792758 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:09.801587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7170fab-51e9-439c-9d82-4733e341b875 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:09.835508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41663111-ea0a-436e-a216-5dd42b8c83ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:09.836258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41663111-ea0a-436e-a216-5dd42b8c83ae None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:09.836544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41663111-ea0a-436e-a216-5dd42b8c83ae None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:09.836817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41663111-ea0a-436e-a216-5dd42b8c83ae None admin] PATH_INFO: `/v3/services/image` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:09.837475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41663111-ea0a-436e-a216-5dd42b8c83ae None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:09.838843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41663111-ea0a-436e-a216-5dd42b8c83ae None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-41663111-ea0a-436e-a216-5dd42b8c83ae None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:09.851143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:09.852227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:09.853092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 533/1061] 10.222.0.64 () {56 vars in 1105 bytes} [Thu Jul 14 17:39:09 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:09.858810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d998caa-2482-4fc8-9808-be94bd352fd6 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:09.869103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:09.963319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d998caa-2482-4fc8-9808-be94bd352fd6 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:09.998802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277fedb1-1b55-4d01-88cf-656ff90b369f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:09.999723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fedb1-1b55-4d01-88cf-656ff90b369f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:10.000032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fedb1-1b55-4d01-88cf-656ff90b369f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:10.000311 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277fedb1-1b55-4d01-88cf-656ff90b369f None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:10.001016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fedb1-1b55-4d01-88cf-656ff90b369f None admin] RBAC: Adding query filter params (name=image) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:10.001321 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fedb1-1b55-4d01-88cf-656ff90b369f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:10.002685 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277fedb1-1b55-4d01-88cf-656ff90b369f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:10.012157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 529/1062] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:10.013286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41663111-ea0a-436e-a216-5dd42b8c83ae None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:10.114788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41663111-ea0a-436e-a216-5dd42b8c83ae None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:10.152361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f470e-eb7e-48a6-b7ee-17a016f7a091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:10.153023 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f470e-eb7e-48a6-b7ee-17a016f7a091 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:10.153305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f470e-eb7e-48a6-b7ee-17a016f7a091 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:10.153579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f470e-eb7e-48a6-b7ee-17a016f7a091 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:10.154287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f470e-eb7e-48a6-b7ee-17a016f7a091 None admin] RBAC: Adding query filter params (type=image) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:10.154596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f470e-eb7e-48a6-b7ee-17a016f7a091 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:10.156616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f470e-eb7e-48a6-b7ee-17a016f7a091 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:10.161798 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 534/1063] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:10.167271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277fedb1-1b55-4d01-88cf-656ff90b369f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:10.244824 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:10.277086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277fedb1-1b55-4d01-88cf-656ff90b369f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:10.312588 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbec6c9-a5b4-4d14-b570-079232fd1d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:10.313306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbec6c9-a5b4-4d14-b570-079232fd1d58 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:10.313598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbec6c9-a5b4-4d14-b570-079232fd1d58 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:10.313878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbec6c9-a5b4-4d14-b570-079232fd1d58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:10.314629 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbec6c9-a5b4-4d14-b570-079232fd1d58 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:10.316022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbec6c9-a5b4-4d14-b570-079232fd1d58 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:10.343954 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 530/1064] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Jul 14 17:39:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:39:10.694524 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbefdc-b656-4361-a13a-a42f394fae15 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:10.694858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbefdc-b656-4361-a13a-a42f394fae15 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:10.695084 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbefdc-b656-4361-a13a-a42f394fae15 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:10.696531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 535/1065] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:39:10.701670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152cf1c-1bc9-4e91-a909-b035de8f5ae3 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:10.702117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152cf1c-1bc9-4e91-a909-b035de8f5ae3 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:10.702284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f152cf1c-1bc9-4e91-a909-b035de8f5ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:10.719317 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f152cf1c-1bc9-4e91-a909-b035de8f5ae3 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:10.720771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f152cf1c-1bc9-4e91-a909-b035de8f5ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:10.799458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 531/1066] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:10.822877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbefdc-b656-4361-a13a-a42f394fae15 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:10.851887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:10.937821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbefdc-b656-4361-a13a-a42f394fae15 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:10.974125 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36692f-53fb-426b-8e1e-720aab84a35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:10.974926 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36692f-53fb-426b-8e1e-720aab84a35f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:10.975212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36692f-53fb-426b-8e1e-720aab84a35f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:10.975489 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36692f-53fb-426b-8e1e-720aab84a35f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:10.986493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36692f-53fb-426b-8e1e-720aab84a35f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-1c36692f-53fb-426b-8e1e-720aab84a35f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:39:10.994218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:10.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:10.996152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:10.996152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:10.996152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:10.996152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:10.996152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:39:10.996152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:39:10.996152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:39:10.996152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:39:10.996152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:39:10.996152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:39:10.996152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:39:10.996152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:10.996152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 536/1067] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:39:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:11.002224 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152cf1c-1bc9-4e91-a909-b035de8f5ae3 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:11.058999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:11.107671 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f152cf1c-1bc9-4e91-a909-b035de8f5ae3 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:11.147496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4274bfb0-dab0-4b16-ac90-6ef94ff2730f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:11.148215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274bfb0-dab0-4b16-ac90-6ef94ff2730f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:11.148503 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274bfb0-dab0-4b16-ac90-6ef94ff2730f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:11.148782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4274bfb0-dab0-4b16-ac90-6ef94ff2730f None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:11.149534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274bfb0-dab0-4b16-ac90-6ef94ff2730f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:11.149845 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274bfb0-dab0-4b16-ac90-6ef94ff2730f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:11.151328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4274bfb0-dab0-4b16-ac90-6ef94ff2730f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:11.165127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 532/1068] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:39:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:11.166366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36692f-53fb-426b-8e1e-720aab84a35f None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:11.255114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:11.273102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36692f-53fb-426b-8e1e-720aab84a35f None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:11.306769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de986964-fa51-4845-8048-06f6d53328fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:11.307647 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de986964-fa51-4845-8048-06f6d53328fe None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:11.307951 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de986964-fa51-4845-8048-06f6d53328fe None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:11.308222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de986964-fa51-4845-8048-06f6d53328fe None admin] PATH_INFO: `/v3/users/84d848cc2d5d4169a4cf4afcd75747b6` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:11.321735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89102-55e7-4657-a88a-75f47a9dd33e None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:11.322016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89102-55e7-4657-a88a-75f47a9dd33e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:11.322303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b89102-55e7-4657-a88a-75f47a9dd33e None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:11.323099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de986964-fa51-4845-8048-06f6d53328fe None admin] RBAC: Authorizing `identity:get_user(user_id=84d848cc2d5d4169a4cf4afcd75747b6)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:11.323832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 533/1069] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:11.324629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de986964-fa51-4845-8048-06f6d53328fe None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:11.331599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 537/1070] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:39:11 2022] GET /identity/v3/users/84d848cc2d5d4169a4cf4afcd75747b6 => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:11.332816 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de986964-fa51-4845-8048-06f6d53328fe None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:11.333202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78446b12-90a6-46d0-8c7b-aa226636efd5 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:11.333478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78446b12-90a6-46d0-8c7b-aa226636efd5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:11.333756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78446b12-90a6-46d0-8c7b-aa226636efd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:11.351196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-78446b12-90a6-46d0-8c7b-aa226636efd5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:11.352559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-78446b12-90a6-46d0-8c7b-aa226636efd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:11.438671 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 534/1071] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:11.448983 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de986964-fa51-4845-8048-06f6d53328fe None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:11.462426 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78446b12-90a6-46d0-8c7b-aa226636efd5 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:11.498844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7280e-978a-471a-9ca9-954460891f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:11.499910 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7280e-978a-471a-9ca9-954460891f48 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:11.500252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7280e-978a-471a-9ca9-954460891f48 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:11.500533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f7280e-978a-471a-9ca9-954460891f48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:11.515090 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:11.520912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:11.582802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78446b12-90a6-46d0-8c7b-aa226636efd5 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:11.587028 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7280e-978a-471a-9ca9-954460891f48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:11.588504 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f7280e-978a-471a-9ca9-954460891f48 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:11.619553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbbfd9-c5d6-45a0-bf03-0a6b64b4f0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:11.620461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbbfd9-c5d6-45a0-bf03-0a6b64b4f0f0 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:11.620748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbbfd9-c5d6-45a0-bf03-0a6b64b4f0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:11.621027 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbbfd9-c5d6-45a0-bf03-0a6b64b4f0f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:11.632526 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fbbfd9-c5d6-45a0-bf03-0a6b64b4f0f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-b1fbbfd9-c5d6-45a0-bf03-0a6b64b4f0f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:39:11.640968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:11.641911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:11.642759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:11.642759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:11.642759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:11.642759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:11.642759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:39:11.642759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:39:11.642759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:39:11.642759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:39:11.642759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:39:11.642759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:39:11.642759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 17:39:11.642759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:11.642759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 535/1072] 10.222.0.64 () {56 vars in 1120 bytes} [Thu Jul 14 17:39:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:39:11.648142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbbfd9-c5d6-45a0-bf03-0a6b64b4f0f0 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:11.704630 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 538/1073] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:39:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:39:11.710519 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7280e-978a-471a-9ca9-954460891f48 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:11.762154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:11.769507 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fbbfd9-c5d6-45a0-bf03-0a6b64b4f0f0 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:11.776979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:11.811718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd790fc-fce2-4708-9c55-d27f063aefbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:11.812705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd790fc-fce2-4708-9c55-d27f063aefbb None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:11.813030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd790fc-fce2-4708-9c55-d27f063aefbb None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:11.813251 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd790fc-fce2-4708-9c55-d27f063aefbb None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:11.814020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd790fc-fce2-4708-9c55-d27f063aefbb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:11.814330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd790fc-fce2-4708-9c55-d27f063aefbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:11.815713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd790fc-fce2-4708-9c55-d27f063aefbb None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:11.825037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 536/1074] 10.222.0.64 () {56 vars in 1115 bytes} [Thu Jul 14 17:39:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:11.831673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd790fc-fce2-4708-9c55-d27f063aefbb None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:11.833334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f7280e-978a-471a-9ca9-954460891f48 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:11.873326 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55a7d5-2d1c-4a0e-8737-c29d87aa5461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:11.875063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55a7d5-2d1c-4a0e-8737-c29d87aa5461 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:11.875505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55a7d5-2d1c-4a0e-8737-c29d87aa5461 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:11.875846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55a7d5-2d1c-4a0e-8737-c29d87aa5461 None admin] PATH_INFO: `/v3/projects/45139be5b6bb456e8323152cec5e3744` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:11.877370 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55a7d5-2d1c-4a0e-8737-c29d87aa5461 None admin] RBAC: Authorizing `identity:get_project(project_id=45139be5b6bb456e8323152cec5e3744)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:11.878976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55a7d5-2d1c-4a0e-8737-c29d87aa5461 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:11.881216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 539/1075] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:39:11 2022] GET /identity/v3/projects/45139be5b6bb456e8323152cec5e3744 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:11.887112 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55a7d5-2d1c-4a0e-8737-c29d87aa5461 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:11.955300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd790fc-fce2-4708-9c55-d27f063aefbb None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:11.994350 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411874a-605e-49e3-b88f-755cedf15195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:11.995173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411874a-605e-49e3-b88f-755cedf15195 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:11.995464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411874a-605e-49e3-b88f-755cedf15195 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:11.995746 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0411874a-605e-49e3-b88f-755cedf15195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:12.007960 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411874a-605e-49e3-b88f-755cedf15195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:12.009475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0411874a-605e-49e3-b88f-755cedf15195 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:12.013911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55a7d5-2d1c-4a0e-8737-c29d87aa5461 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:12.027750 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-0411874a-605e-49e3-b88f-755cedf15195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:39:12.028093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:12.028966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:12.029888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:39:12.029888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:39:12.029888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:39:12.029888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:12.032248 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 537/1076] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:12.036238 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411874a-605e-49e3-b88f-755cedf15195 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:12.058218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d8c0d-c00a-473f-bb54-bf6921fbe723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:12.059232 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d8c0d-c00a-473f-bb54-bf6921fbe723 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:12.059681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d8c0d-c00a-473f-bb54-bf6921fbe723 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:12.060170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d8c0d-c00a-473f-bb54-bf6921fbe723 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:12.061131 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d8c0d-c00a-473f-bb54-bf6921fbe723 None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=45139be5b6bb456e8323152cec5e3744, user.id=84d848cc2d5d4169a4cf4afcd75747b6) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:12.062075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d8c0d-c00a-473f-bb54-bf6921fbe723 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:12.063549 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d8c0d-c00a-473f-bb54-bf6921fbe723 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:12.067101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:12.069882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 540/1077] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:39:11 2022] GET /identity/v3/role_assignments?user.id=84d848cc2d5d4169a4cf4afcd75747b6&scope.project.id=45139be5b6bb456e8323152cec5e3744&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:12.188909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0411874a-605e-49e3-b88f-755cedf15195 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:12.242072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448e605-7536-4485-ab49-2dc81799db00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:12.242757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448e605-7536-4485-ab49-2dc81799db00 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:12.243046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448e605-7536-4485-ab49-2dc81799db00 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:12.243326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448e605-7536-4485-ab49-2dc81799db00 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:12.244087 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448e605-7536-4485-ab49-2dc81799db00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:12.244411 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448e605-7536-4485-ab49-2dc81799db00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:12.245779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448e605-7536-4485-ab49-2dc81799db00 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:12.259595 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 538/1078] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:39:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:12.267041 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d8c0d-c00a-473f-bb54-bf6921fbe723 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:12.392004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d8c0d-c00a-473f-bb54-bf6921fbe723 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:12.432438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc424711-45ba-4d78-8325-4a216212df6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:12.433236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc424711-45ba-4d78-8325-4a216212df6d None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:12.433526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc424711-45ba-4d78-8325-4a216212df6d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:12.433801 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc424711-45ba-4d78-8325-4a216212df6d None admin] PATH_INFO: `/v3/users/nova` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:12.436373 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90121e5-5344-40e2-9bf5-a87ba2b2d29d None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:12.437208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90121e5-5344-40e2-9bf5-a87ba2b2d29d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:12.437494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90121e5-5344-40e2-9bf5-a87ba2b2d29d None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:12.438886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 539/1079] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:12.442839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc424711-45ba-4d78-8325-4a216212df6d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:12.444548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc424711-45ba-4d78-8325-4a216212df6d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:12.444761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88013341-630a-425d-a539-2ad0aa5f2333 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:12.444990 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88013341-630a-425d-a539-2ad0aa5f2333 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:12.445265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88013341-630a-425d-a539-2ad0aa5f2333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:12.461719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-88013341-630a-425d-a539-2ad0aa5f2333 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:12.462921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-88013341-630a-425d-a539-2ad0aa5f2333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-fc424711-45ba-4d78-8325-4a216212df6d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:39:12.463474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:12.464472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:12.465436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:12.465436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:12.465436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:39:12.465436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:39:12.465436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:39:12.465436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:39:12.465436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:39:12.465436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:39:12.465436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 17:39:12.465436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:12.469793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 541/1080] 10.222.0.64 () {56 vars in 1093 bytes} [Thu Jul 14 17:39:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:12.473696 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc424711-45ba-4d78-8325-4a216212df6d None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:12.517362 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:12.553489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 540/1081] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:39:12.577735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88013341-630a-425d-a539-2ad0aa5f2333 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:12.592984 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:12.604662 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc424711-45ba-4d78-8325-4a216212df6d None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:12.650022 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cb4c5-9db9-4802-89ee-8d691a1f5354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:12.650804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cb4c5-9db9-4802-89ee-8d691a1f5354 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:12.650932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cb4c5-9db9-4802-89ee-8d691a1f5354 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:12.651158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cb4c5-9db9-4802-89ee-8d691a1f5354 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:12.652051 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cb4c5-9db9-4802-89ee-8d691a1f5354 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:12.652367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cb4c5-9db9-4802-89ee-8d691a1f5354 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:12.653802 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cb4c5-9db9-4802-89ee-8d691a1f5354 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:12.677519 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 542/1082] 10.222.0.64 () {56 vars in 1133 bytes} [Thu Jul 14 17:39:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:12.683105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cb4c5-9db9-4802-89ee-8d691a1f5354 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:12.706364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88013341-630a-425d-a539-2ad0aa5f2333 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:12.750702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58d214-f4fc-42d6-83be-e462ee506c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:12.751362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58d214-f4fc-42d6-83be-e462ee506c1e None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:12.751673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58d214-f4fc-42d6-83be-e462ee506c1e None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:12.752489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58d214-f4fc-42d6-83be-e462ee506c1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:12.801184 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:12.815193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cb4c5-9db9-4802-89ee-8d691a1f5354 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:12.846492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58d214-f4fc-42d6-83be-e462ee506c1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:12.848017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58d214-f4fc-42d6-83be-e462ee506c1e None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:12.858352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3507a0e-448a-4c56-b76e-3264c0c679cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:12.859917 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3507a0e-448a-4c56-b76e-3264c0c679cb None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:12.860455 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3507a0e-448a-4c56-b76e-3264c0c679cb None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:12.860913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3507a0e-448a-4c56-b76e-3264c0c679cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:12.867595 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:12.951422 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3507a0e-448a-4c56-b76e-3264c0c679cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:12.953935 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3507a0e-448a-4c56-b76e-3264c0c679cb None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:12.994728 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 541/1083] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:39:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 17:39:13.002623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58d214-f4fc-42d6-83be-e462ee506c1e None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:13.089719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:13.111056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 543/1084] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:39:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:39:13.120133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3507a0e-448a-4c56-b76e-3264c0c679cb None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:13.160286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58d214-f4fc-42d6-83be-e462ee506c1e None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:13.173593 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:13.212421 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c5fad-1f37-45ff-ac43-d77234c5326d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:13.213150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c5fad-1f37-45ff-ac43-d77234c5326d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:13.213435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c5fad-1f37-45ff-ac43-d77234c5326d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:13.213718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c5fad-1f37-45ff-ac43-d77234c5326d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:13.228060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c5fad-1f37-45ff-ac43-d77234c5326d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:13.229541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c5fad-1f37-45ff-ac43-d77234c5326d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-352c5fad-1f37-45ff-ac43-d77234c5326d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:39:13.249274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:13.251177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:13.252898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:39:13.252898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:39:13.252898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:39:13.252898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:13.252898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 542/1085] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:13.256910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c5fad-1f37-45ff-ac43-d77234c5326d None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:13.317528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3507a0e-448a-4c56-b76e-3264c0c679cb None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:13.380290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6fd7e-b5c9-4057-b8bd-0f6370266a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:13.381534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6fd7e-b5c9-4057-b8bd-0f6370266a2e None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:13.382002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6fd7e-b5c9-4057-b8bd-0f6370266a2e None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:13.382482 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d6fd7e-b5c9-4057-b8bd-0f6370266a2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:13.382844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c5fad-1f37-45ff-ac43-d77234c5326d None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:13.401147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6fd7e-b5c9-4057-b8bd-0f6370266a2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:13.402644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d6fd7e-b5c9-4057-b8bd-0f6370266a2e None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-83d6fd7e-b5c9-4057-b8bd-0f6370266a2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:39:13.423625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:13.433307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:13.435677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:39:13.435677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:39:13.435677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:39:13.435677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:13.435677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 544/1086] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:39:13.435677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6fd7e-b5c9-4057-b8bd-0f6370266a2e None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:13.448155 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bf883-f4d6-4ada-a67d-a95c3cb2838c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:13.449918 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bf883-f4d6-4ada-a67d-a95c3cb2838c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:13.449918 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bf883-f4d6-4ada-a67d-a95c3cb2838c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:13.450252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bf883-f4d6-4ada-a67d-a95c3cb2838c None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:13.451588 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bf883-f4d6-4ada-a67d-a95c3cb2838c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:13.452231 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bf883-f4d6-4ada-a67d-a95c3cb2838c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:13.455259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bf883-f4d6-4ada-a67d-a95c3cb2838c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:13.474537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 543/1087] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:13.537498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:13.649637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d6fd7e-b5c9-4057-b8bd-0f6370266a2e None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:13.701768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e53e0-8054-47ea-8714-a7c17fdf9f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:13.702476 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e53e0-8054-47ea-8714-a7c17fdf9f6c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:13.702758 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e53e0-8054-47ea-8714-a7c17fdf9f6c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:13.703033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e53e0-8054-47ea-8714-a7c17fdf9f6c None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:13.703777 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e53e0-8054-47ea-8714-a7c17fdf9f6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:13.704126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e53e0-8054-47ea-8714-a7c17fdf9f6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:13.705513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e53e0-8054-47ea-8714-a7c17fdf9f6c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:13.723250 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 545/1088] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:13.736002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bf883-f4d6-4ada-a67d-a95c3cb2838c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:13.794080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:13.941067 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bf883-f4d6-4ada-a67d-a95c3cb2838c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:14.001864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199fa05-e025-4784-b97d-e36e51d15dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:14.003127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199fa05-e025-4784-b97d-e36e51d15dee None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:14.003622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199fa05-e025-4784-b97d-e36e51d15dee None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:14.004173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199fa05-e025-4784-b97d-e36e51d15dee None admin] PATH_INFO: `/v3/projects/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:14.017790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199fa05-e025-4784-b97d-e36e51d15dee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:14.020494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199fa05-e025-4784-b97d-e36e51d15dee None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-b199fa05-e025-4784-b97d-e36e51d15dee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:39:14.049506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:14.051105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:14.063189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:39:14.063189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:39:14.063189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:39:14.063189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:39:14.063189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:39:14.063189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:39:14.063189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:39:14.063189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:14.063189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 544/1089] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:39:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:14.065415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e53e0-8054-47ea-8714-a7c17fdf9f6c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:14.207173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e53e0-8054-47ea-8714-a7c17fdf9f6c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:14.211210 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:14.247445 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636ecb4-8881-4dc4-90e0-10b8d68651b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:14.248331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636ecb4-8881-4dc4-90e0-10b8d68651b8 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:14.248737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636ecb4-8881-4dc4-90e0-10b8d68651b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:14.249643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636ecb4-8881-4dc4-90e0-10b8d68651b8 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:14.250823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636ecb4-8881-4dc4-90e0-10b8d68651b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:14.251259 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636ecb4-8881-4dc4-90e0-10b8d68651b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:14.252953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636ecb4-8881-4dc4-90e0-10b8d68651b8 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:14.267322 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 546/1090] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:14.274446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199fa05-e025-4784-b97d-e36e51d15dee None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:14.286241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80810b24-0cf7-4749-a097-1cc3f8e9d230 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:14.286523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80810b24-0cf7-4749-a097-1cc3f8e9d230 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:14.286803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80810b24-0cf7-4749-a097-1cc3f8e9d230 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:14.288246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 547/1091] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:14.293769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df998e-e9cc-43ff-8b90-033dac3c5a34 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:14.294074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df998e-e9cc-43ff-8b90-033dac3c5a34 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:14.294357 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df998e-e9cc-43ff-8b90-033dac3c5a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:14.310768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-17df998e-e9cc-43ff-8b90-033dac3c5a34 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:14.312540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-17df998e-e9cc-43ff-8b90-033dac3c5a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:14.396014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:14.399986 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 548/1092] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:39:14.429089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199fa05-e025-4784-b97d-e36e51d15dee None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:14.439394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df998e-e9cc-43ff-8b90-033dac3c5a34 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:14.477654 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d8d58-25cc-4304-b58c-2dfbeffb3129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:14.478519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d8d58-25cc-4304-b58c-2dfbeffb3129 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:14.478865 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d8d58-25cc-4304-b58c-2dfbeffb3129 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:14.479208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203d8d58-25cc-4304-b58c-2dfbeffb3129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:14.480249 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d8d58-25cc-4304-b58c-2dfbeffb3129 None admin] RBAC: Adding query filter params (role.id=086a0517b6e44f8aa33468ec2e3135a7, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=6423f54152064b909d03c0908d636993) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:14.480682 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d8d58-25cc-4304-b58c-2dfbeffb3129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:14.482442 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203d8d58-25cc-4304-b58c-2dfbeffb3129 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:14.490023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 545/1093] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:39:14 2022] GET /identity/v3/role_assignments?user.id=6423f54152064b909d03c0908d636993&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=086a0517b6e44f8aa33468ec2e3135a7 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:14.556089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:14.564615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df998e-e9cc-43ff-8b90-033dac3c5a34 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:14.606860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f862f-6de8-4309-8171-9c735d6e894c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:14.607805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f862f-6de8-4309-8171-9c735d6e894c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:14.608213 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f862f-6de8-4309-8171-9c735d6e894c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:14.609092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f862f-6de8-4309-8171-9c735d6e894c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:14.621497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f862f-6de8-4309-8171-9c735d6e894c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-1f2f862f-6de8-4309-8171-9c735d6e894c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:39:14.630381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:14.632506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:14.633589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:14.633589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:14.633589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:14.633589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:14.633589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:39:14.633589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:39:14.633589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:39:14.633589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:39:14.633589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:39:14.633589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:39:14.633589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:39:14.633589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:14.633589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 549/1094] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:39:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:14.639292 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d8d58-25cc-4304-b58c-2dfbeffb3129 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:14.774215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203d8d58-25cc-4304-b58c-2dfbeffb3129 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:14.821505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa04678-4092-456e-a7e7-9dd919439801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:14.822529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa04678-4092-456e-a7e7-9dd919439801 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:14.823002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa04678-4092-456e-a7e7-9dd919439801 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:14.823398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa04678-4092-456e-a7e7-9dd919439801 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:14.824616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa04678-4092-456e-a7e7-9dd919439801 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:14.825034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa04678-4092-456e-a7e7-9dd919439801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:14.827410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa04678-4092-456e-a7e7-9dd919439801 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:14.838932 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 546/1095] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:39:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:14.845104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f862f-6de8-4309-8171-9c735d6e894c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:14.992166 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f862f-6de8-4309-8171-9c735d6e894c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:15.033757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37332d-1efb-411e-801b-16f4e8b4a80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:15.034646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37332d-1efb-411e-801b-16f4e8b4a80e None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:15.034933 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37332d-1efb-411e-801b-16f4e8b4a80e None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:15.035210 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37332d-1efb-411e-801b-16f4e8b4a80e None admin] PATH_INFO: `/v3/users/84d848cc2d5d4169a4cf4afcd75747b6` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:15.036498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37332d-1efb-411e-801b-16f4e8b4a80e None admin] RBAC: Authorizing `identity:get_user(user_id=84d848cc2d5d4169a4cf4afcd75747b6)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:15.037948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37332d-1efb-411e-801b-16f4e8b4a80e None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:15.039634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 550/1096] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:39:14 2022] GET /identity/v3/users/84d848cc2d5d4169a4cf4afcd75747b6 => generated 285 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:15.048641 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa04678-4092-456e-a7e7-9dd919439801 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:15.063765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:15.184919 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa04678-4092-456e-a7e7-9dd919439801 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:15.233262 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43a840-e4ff-4229-83d9-153380038ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:15.234823 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43a840-e4ff-4229-83d9-153380038ee0 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:15.235985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43a840-e4ff-4229-83d9-153380038ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:15.235985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43a840-e4ff-4229-83d9-153380038ee0 None admin] PATH_INFO: `/v3/projects/45139be5b6bb456e8323152cec5e3744` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:15.237378 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43a840-e4ff-4229-83d9-153380038ee0 None admin] RBAC: Authorizing `identity:get_project(project_id=45139be5b6bb456e8323152cec5e3744)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:15.239189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43a840-e4ff-4229-83d9-153380038ee0 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:15.242384 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 547/1097] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:39:15 2022] GET /identity/v3/projects/45139be5b6bb456e8323152cec5e3744 => generated 311 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:15.250531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37332d-1efb-411e-801b-16f4e8b4a80e None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:15.284807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:15.400537 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37332d-1efb-411e-801b-16f4e8b4a80e None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:15.462315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f6b75-f66c-4fc8-89e2-ebf3cb09a05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:15.463881 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f6b75-f66c-4fc8-89e2-ebf3cb09a05d None admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:15.464398 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f6b75-f66c-4fc8-89e2-ebf3cb09a05d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:15.464879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f6b75-f66c-4fc8-89e2-ebf3cb09a05d None admin] PATH_INFO: `/v3/projects/45139be5b6bb456e8323152cec5e3744/users/84d848cc2d5d4169a4cf4afcd75747b6/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:15.468547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f6b75-f66c-4fc8-89e2-ebf3cb09a05d None admin] RBAC: Authorizing `identity:create_grant(project_id=45139be5b6bb456e8323152cec5e3744, user_id=84d848cc2d5d4169a4cf4afcd75747b6, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:15.471305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f6b75-f66c-4fc8-89e2-ebf3cb09a05d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:15.494586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 551/1098] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:39:15 2022] PUT /identity/v3/projects/45139be5b6bb456e8323152cec5e3744/users/84d848cc2d5d4169a4cf4afcd75747b6/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:39:17.590599 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89c821-24a5-421c-b1ae-e0cca3381020 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:17.591214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89c821-24a5-421c-b1ae-e0cca3381020 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:17.591214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89c821-24a5-421c-b1ae-e0cca3381020 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:17.592776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 548/1099] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:17.598791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13dac9-56d4-4233-a89c-fd0f9711baca None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:17.599356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13dac9-56d4-4233-a89c-fd0f9711baca None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:17.599356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13dac9-56d4-4233-a89c-fd0f9711baca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:17.617300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-ee13dac9-56d4-4233-a89c-fd0f9711baca None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:17.618511 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ee13dac9-56d4-4233-a89c-fd0f9711baca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:17.691142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 552/1100] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:39:17.734782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89c821-24a5-421c-b1ae-e0cca3381020 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:17.823550 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:17.851347 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89c821-24a5-421c-b1ae-e0cca3381020 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:17.889393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4805d-a948-4ced-8198-d8f06f03a162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:17.890285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4805d-a948-4ced-8198-d8f06f03a162 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:17.890572 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4805d-a948-4ced-8198-d8f06f03a162 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:17.890849 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe4805d-a948-4ced-8198-d8f06f03a162 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:17.903897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe4805d-a948-4ced-8198-d8f06f03a162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-4fe4805d-a948-4ced-8198-d8f06f03a162 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:39:17.912271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:17.914046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:17.914976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:17.914976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:17.914976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:17.914976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:17.914976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:39:17.914976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:39:17.914976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:39:17.914976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:39:17.914976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:39:17.914976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:39:17.914976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:39:17.914976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:17.914976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 549/1101] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:39:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:17.923943 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13dac9-56d4-4233-a89c-fd0f9711baca None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:17.939753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:18.037691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13dac9-56d4-4233-a89c-fd0f9711baca None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:18.076118 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14a0ce-94d5-4fd1-aec1-5f03b0db2cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:18.076810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14a0ce-94d5-4fd1-aec1-5f03b0db2cb8 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:18.077104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14a0ce-94d5-4fd1-aec1-5f03b0db2cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:18.078041 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14a0ce-94d5-4fd1-aec1-5f03b0db2cb8 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:18.078962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14a0ce-94d5-4fd1-aec1-5f03b0db2cb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:18.079434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14a0ce-94d5-4fd1-aec1-5f03b0db2cb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:18.080864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14a0ce-94d5-4fd1-aec1-5f03b0db2cb8 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:18.090807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 553/1102] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:39:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:18.100571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4805d-a948-4ced-8198-d8f06f03a162 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:18.222345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe4805d-a948-4ced-8198-d8f06f03a162 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:18.265964 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1d8d0-0538-435a-b504-9a239c70a5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:18.266863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1d8d0-0538-435a-b504-9a239c70a5b9 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:18.267148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1d8d0-0538-435a-b504-9a239c70a5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:18.267429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1d8d0-0538-435a-b504-9a239c70a5b9 None admin] PATH_INFO: `/v3/users/84d848cc2d5d4169a4cf4afcd75747b6` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:18.268785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1d8d0-0538-435a-b504-9a239c70a5b9 None admin] RBAC: Authorizing `identity:get_user(user_id=84d848cc2d5d4169a4cf4afcd75747b6)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:18.270280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c1d8d0-0538-435a-b504-9a239c70a5b9 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:18.272388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 550/1103] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:39:18 2022] GET /identity/v3/users/84d848cc2d5d4169a4cf4afcd75747b6 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:18.278830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14a0ce-94d5-4fd1-aec1-5f03b0db2cb8 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:18.361976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:18.393888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14a0ce-94d5-4fd1-aec1-5f03b0db2cb8 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:18.432503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30cf0b-9bcb-47dc-986c-9a21652ddaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:18.433193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30cf0b-9bcb-47dc-986c-9a21652ddaaa None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:18.433480 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30cf0b-9bcb-47dc-986c-9a21652ddaaa None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:18.433760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30cf0b-9bcb-47dc-986c-9a21652ddaaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:18.516207 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30cf0b-9bcb-47dc-986c-9a21652ddaaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:18.517712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30cf0b-9bcb-47dc-986c-9a21652ddaaa None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:18.603230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:18.632519 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 554/1104] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:39:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 17:39:18.639144 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1d8d0-0538-435a-b504-9a239c70a5b9 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:18.643403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:18.750928 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1d8d0-0538-435a-b504-9a239c70a5b9 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:18.788327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400dc62-8b51-4b14-baf6-2042be94ce89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:18.789200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400dc62-8b51-4b14-baf6-2042be94ce89 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:18.789487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400dc62-8b51-4b14-baf6-2042be94ce89 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:18.789769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400dc62-8b51-4b14-baf6-2042be94ce89 None admin] PATH_INFO: `/v3/projects/45139be5b6bb456e8323152cec5e3744` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:18.791059 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400dc62-8b51-4b14-baf6-2042be94ce89 None admin] RBAC: Authorizing `identity:get_project(project_id=45139be5b6bb456e8323152cec5e3744)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:18.792579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400dc62-8b51-4b14-baf6-2042be94ce89 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:18.794425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 551/1105] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:39:18 2022] GET /identity/v3/projects/45139be5b6bb456e8323152cec5e3744 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:18.801685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30cf0b-9bcb-47dc-986c-9a21652ddaaa None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:18.911873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30cf0b-9bcb-47dc-986c-9a21652ddaaa None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:18.949256 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53a741-cd2d-445d-af67-168a40b20f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:18.950002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53a741-cd2d-445d-af67-168a40b20f84 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:18.950389 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53a741-cd2d-445d-af67-168a40b20f84 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:18.950714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53a741-cd2d-445d-af67-168a40b20f84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:18.951623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53a741-cd2d-445d-af67-168a40b20f84 None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=45139be5b6bb456e8323152cec5e3744, user.id=84d848cc2d5d4169a4cf4afcd75747b6) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:18.952016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53a741-cd2d-445d-af67-168a40b20f84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:18.954134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53a741-cd2d-445d-af67-168a40b20f84 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:18.961034 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 555/1106] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:39:18 2022] GET /identity/v3/role_assignments?user.id=84d848cc2d5d4169a4cf4afcd75747b6&scope.project.id=45139be5b6bb456e8323152cec5e3744&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:21.088204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e30e22-2b7a-4f0e-884b-5f17def29726 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:21.089052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e30e22-2b7a-4f0e-884b-5f17def29726 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:21.089052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e30e22-2b7a-4f0e-884b-5f17def29726 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:21.090316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 552/1107] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:21.097240 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffd051-17f4-4b6d-949b-7384f4ac14ac None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:21.097988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffd051-17f4-4b6d-949b-7384f4ac14ac None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:21.097988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ffd051-17f4-4b6d-949b-7384f4ac14ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:21.115178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-99ffd051-17f4-4b6d-949b-7384f4ac14ac None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:21.116442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-99ffd051-17f4-4b6d-949b-7384f4ac14ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:21.150717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:21.192954 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 556/1108] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:21.218608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e30e22-2b7a-4f0e-884b-5f17def29726 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:21.296779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:21.330560 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e30e22-2b7a-4f0e-884b-5f17def29726 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:21.373490 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d545363-753d-4028-8090-897cf7ee18ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:21.376476 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d545363-753d-4028-8090-897cf7ee18ed None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:21.376980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d545363-753d-4028-8090-897cf7ee18ed None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:21.377468 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d545363-753d-4028-8090-897cf7ee18ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:21.379544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d545363-753d-4028-8090-897cf7ee18ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:21.382299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d545363-753d-4028-8090-897cf7ee18ed None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:21.385371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 553/1109] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:21.391523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffd051-17f4-4b6d-949b-7384f4ac14ac None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:21.506267 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ffd051-17f4-4b6d-949b-7384f4ac14ac None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:21.541141 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0077e09f-bb49-48ae-8d06-77708ce09447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:21.541823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077e09f-bb49-48ae-8d06-77708ce09447 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:21.542109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077e09f-bb49-48ae-8d06-77708ce09447 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:21.542387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077e09f-bb49-48ae-8d06-77708ce09447 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:21.543129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077e09f-bb49-48ae-8d06-77708ce09447 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:21.544622 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0077e09f-bb49-48ae-8d06-77708ce09447 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:21.574907 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 557/1110] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:39:21 2022] POST /identity/v3/users => generated 286 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:39:23.725082 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc1bc2-424f-4c1e-88d3-59e5973b05da None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:23.725689 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc1bc2-424f-4c1e-88d3-59e5973b05da None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:23.725689 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc1bc2-424f-4c1e-88d3-59e5973b05da None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:23.727014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 554/1111] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:23.733329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd0853-50ec-4423-b93f-26ff96865599 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:23.733936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd0853-50ec-4423-b93f-26ff96865599 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:23.733936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd0853-50ec-4423-b93f-26ff96865599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:23.752582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-a4bd0853-50ec-4423-b93f-26ff96865599 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:23.753942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a4bd0853-50ec-4423-b93f-26ff96865599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:23.773350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:23.831505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 558/1112] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:39:23.876365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc1bc2-424f-4c1e-88d3-59e5973b05da None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:24.014078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc1bc2-424f-4c1e-88d3-59e5973b05da None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:24.051816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d914f-598f-41cc-aafe-3fc36bbab41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:24.052729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d914f-598f-41cc-aafe-3fc36bbab41c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:24.053014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d914f-598f-41cc-aafe-3fc36bbab41c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:24.053292 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d914f-598f-41cc-aafe-3fc36bbab41c None admin] PATH_INFO: `/v3/roles/8427479f632a4da3949971f58b438c7f` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:24.054837 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d914f-598f-41cc-aafe-3fc36bbab41c None admin] RBAC: Authorizing `identity:get_role(role_id=8427479f632a4da3949971f58b438c7f)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:24.056258 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d914f-598f-41cc-aafe-3fc36bbab41c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:24.057669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 555/1113] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:39:23 2022] GET /identity/v3/roles/8427479f632a4da3949971f58b438c7f => generated 223 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:24.063787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd0853-50ec-4423-b93f-26ff96865599 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:24.174164 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd0853-50ec-4423-b93f-26ff96865599 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:24.213174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36713c-67fc-406c-ab44-3fbc5a303be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:24.214061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36713c-67fc-406c-ab44-3fbc5a303be8 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:24.214346 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36713c-67fc-406c-ab44-3fbc5a303be8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:24.214622 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d36713c-67fc-406c-ab44-3fbc5a303be8 None admin] PATH_INFO: `/v3/users/a63ea4c1aa66488fa15e7c2e1904cdc3` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:24.229790 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36713c-67fc-406c-ab44-3fbc5a303be8 None admin] RBAC: Authorizing `identity:get_user(user_id=a63ea4c1aa66488fa15e7c2e1904cdc3)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:24.231263 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d36713c-67fc-406c-ab44-3fbc5a303be8 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:24.233084 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 559/1114] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:39:24 2022] GET /identity/v3/users/a63ea4c1aa66488fa15e7c2e1904cdc3 => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:24.239755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d914f-598f-41cc-aafe-3fc36bbab41c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:24.252126 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:24.354003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d914f-598f-41cc-aafe-3fc36bbab41c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:24.396202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f713f-0645-4976-8a4d-68d9de93bc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:24.396863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f713f-0645-4976-8a4d-68d9de93bc1a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:24.397146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f713f-0645-4976-8a4d-68d9de93bc1a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:24.397426 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f713f-0645-4976-8a4d-68d9de93bc1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:24.476957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f713f-0645-4976-8a4d-68d9de93bc1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:24.478490 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f713f-0645-4976-8a4d-68d9de93bc1a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:24.491645 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:24.596858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 556/1115] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:39:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 17:39:24.603313 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36713c-67fc-406c-ab44-3fbc5a303be8 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:24.621190 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:24.716836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d36713c-67fc-406c-ab44-3fbc5a303be8 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:24.754911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c27ed-12e3-4e30-965b-2e5a3bf6b8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:24.756010 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c27ed-12e3-4e30-965b-2e5a3bf6b8b7 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:24.756171 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c27ed-12e3-4e30-965b-2e5a3bf6b8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:24.756340 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c27ed-12e3-4e30-965b-2e5a3bf6b8b7 None admin] PATH_INFO: `/v3/projects/45139be5b6bb456e8323152cec5e3744` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:24.757553 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c27ed-12e3-4e30-965b-2e5a3bf6b8b7 None admin] RBAC: Authorizing `identity:get_project(project_id=45139be5b6bb456e8323152cec5e3744)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:24.759007 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c27ed-12e3-4e30-965b-2e5a3bf6b8b7 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:24.760887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 560/1116] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:39:24 2022] GET /identity/v3/projects/45139be5b6bb456e8323152cec5e3744 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:24.768152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f713f-0645-4976-8a4d-68d9de93bc1a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:24.874971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f713f-0645-4976-8a4d-68d9de93bc1a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:24.885377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:24.917397 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474387b8-9060-4816-bd4e-9439ecf8272c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:24.918506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474387b8-9060-4816-bd4e-9439ecf8272c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:24.918945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474387b8-9060-4816-bd4e-9439ecf8272c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:24.919424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474387b8-9060-4816-bd4e-9439ecf8272c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:24.921025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474387b8-9060-4816-bd4e-9439ecf8272c None admin] RBAC: Adding query filter params (role.id=8427479f632a4da3949971f58b438c7f, scope.project.id=45139be5b6bb456e8323152cec5e3744, user.id=a63ea4c1aa66488fa15e7c2e1904cdc3) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:24.921569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474387b8-9060-4816-bd4e-9439ecf8272c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:24.924254 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474387b8-9060-4816-bd4e-9439ecf8272c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:24.934626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 557/1117] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:39:24 2022] GET /identity/v3/role_assignments?user.id=a63ea4c1aa66488fa15e7c2e1904cdc3&scope.project.id=45139be5b6bb456e8323152cec5e3744&role.id=8427479f632a4da3949971f58b438c7f => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:27.029656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755da57-d6a2-4d73-8a5e-6dcaa3c0307d None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:27.030258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755da57-d6a2-4d73-8a5e-6dcaa3c0307d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:27.030258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755da57-d6a2-4d73-8a5e-6dcaa3c0307d None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:27.032172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 561/1118] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:27.038093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca04ec-0d14-4e3c-aea4-52e09c70ffa5 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:27.038630 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca04ec-0d14-4e3c-aea4-52e09c70ffa5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:27.038770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ca04ec-0d14-4e3c-aea4-52e09c70ffa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:27.056585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-08ca04ec-0d14-4e3c-aea4-52e09c70ffa5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:27.057864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-08ca04ec-0d14-4e3c-aea4-52e09c70ffa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:27.132458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 558/1119] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:39:27.156465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755da57-d6a2-4d73-8a5e-6dcaa3c0307d None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:27.245286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:27.268047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755da57-d6a2-4d73-8a5e-6dcaa3c0307d None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:27.305187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9867100-5e84-4560-b052-fea0fcdafcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:27.305954 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9867100-5e84-4560-b052-fea0fcdafcea None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:27.306239 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9867100-5e84-4560-b052-fea0fcdafcea None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:27.306517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9867100-5e84-4560-b052-fea0fcdafcea None admin] PATH_INFO: `/v3/roles/8427479f632a4da3949971f58b438c7f` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:27.308057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9867100-5e84-4560-b052-fea0fcdafcea None admin] RBAC: Authorizing `identity:get_role(role_id=8427479f632a4da3949971f58b438c7f)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:27.309447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9867100-5e84-4560-b052-fea0fcdafcea None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:27.310864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 562/1120] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:39:27 2022] GET /identity/v3/roles/8427479f632a4da3949971f58b438c7f => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:27.317514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca04ec-0d14-4e3c-aea4-52e09c70ffa5 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:27.433600 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ca04ec-0d14-4e3c-aea4-52e09c70ffa5 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:27.436411 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:27.473445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca50679-e4f3-413a-82a2-00af3f454028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:27.474352 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca50679-e4f3-413a-82a2-00af3f454028 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:27.474640 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca50679-e4f3-413a-82a2-00af3f454028 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:27.474923 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca50679-e4f3-413a-82a2-00af3f454028 None admin] PATH_INFO: `/v3/users/a63ea4c1aa66488fa15e7c2e1904cdc3` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:27.476180 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca50679-e4f3-413a-82a2-00af3f454028 None admin] RBAC: Authorizing `identity:get_user(user_id=a63ea4c1aa66488fa15e7c2e1904cdc3)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:27.477642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca50679-e4f3-413a-82a2-00af3f454028 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:27.479343 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 559/1121] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:39:27 2022] GET /identity/v3/users/a63ea4c1aa66488fa15e7c2e1904cdc3 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:27.488050 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9867100-5e84-4560-b052-fea0fcdafcea None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:27.602382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9867100-5e84-4560-b052-fea0fcdafcea None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:27.638934 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336dd9e-4d47-46f3-92df-ffb6dc6b35af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:27.639864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dd9e-4d47-46f3-92df-ffb6dc6b35af None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:27.640887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dd9e-4d47-46f3-92df-ffb6dc6b35af None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:27.641224 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336dd9e-4d47-46f3-92df-ffb6dc6b35af None admin] PATH_INFO: `/v3/projects/45139be5b6bb456e8323152cec5e3744` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:27.642521 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336dd9e-4d47-46f3-92df-ffb6dc6b35af None admin] RBAC: Authorizing `identity:get_project(project_id=45139be5b6bb456e8323152cec5e3744)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:27.644081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336dd9e-4d47-46f3-92df-ffb6dc6b35af None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:27.645884 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 563/1122] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:39:27 2022] GET /identity/v3/projects/45139be5b6bb456e8323152cec5e3744 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:27.652190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca50679-e4f3-413a-82a2-00af3f454028 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:27.759373 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca50679-e4f3-413a-82a2-00af3f454028 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:27.798066 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a903c0-ccec-4ea8-99d7-954a49f8d5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:27.798976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a903c0-ccec-4ea8-99d7-954a49f8d5a1 None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:27.799265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a903c0-ccec-4ea8-99d7-954a49f8d5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:27.799546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a903c0-ccec-4ea8-99d7-954a49f8d5a1 None admin] PATH_INFO: `/v3/projects/45139be5b6bb456e8323152cec5e3744/users/a63ea4c1aa66488fa15e7c2e1904cdc3/roles/8427479f632a4da3949971f58b438c7f` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:27.801854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a903c0-ccec-4ea8-99d7-954a49f8d5a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=45139be5b6bb456e8323152cec5e3744, user_id=a63ea4c1aa66488fa15e7c2e1904cdc3, role_id=8427479f632a4da3949971f58b438c7f)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:27.803318 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a903c0-ccec-4ea8-99d7-954a49f8d5a1 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:27.819271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 560/1123] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:39:27 2022] PUT /identity/v3/projects/45139be5b6bb456e8323152cec5e3744/users/a63ea4c1aa66488fa15e7c2e1904cdc3/roles/8427479f632a4da3949971f58b438c7f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:39:29.921578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfdeace-a483-4712-a04c-466ac122fe37 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:29.922415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfdeace-a483-4712-a04c-466ac122fe37 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:29.922569 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfdeace-a483-4712-a04c-466ac122fe37 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:29.924681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 564/1124] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:39:29.930235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd1a34-0aca-4a95-95f6-ea444f3830e1 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:29.930700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd1a34-0aca-4a95-95f6-ea444f3830e1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:29.930825 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd1a34-0aca-4a95-95f6-ea444f3830e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:29.948173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b2dd1a34-0aca-4a95-95f6-ea444f3830e1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:29.949360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b2dd1a34-0aca-4a95-95f6-ea444f3830e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:29.974289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:30.042751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 561/1125] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:30.066471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfdeace-a483-4712-a04c-466ac122fe37 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:30.085327 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:30.176494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfdeace-a483-4712-a04c-466ac122fe37 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:30.228289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e620b9-c6ee-4666-abd0-d0ae3e6c5d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:30.229681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e620b9-c6ee-4666-abd0-d0ae3e6c5d46 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:30.230186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e620b9-c6ee-4666-abd0-d0ae3e6c5d46 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:30.230693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e620b9-c6ee-4666-abd0-d0ae3e6c5d46 None admin] PATH_INFO: `/v3/roles/8427479f632a4da3949971f58b438c7f` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:30.233388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e620b9-c6ee-4666-abd0-d0ae3e6c5d46 None admin] RBAC: Authorizing `identity:get_role(role_id=8427479f632a4da3949971f58b438c7f)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:30.236528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e620b9-c6ee-4666-abd0-d0ae3e6c5d46 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:30.239137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 565/1126] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:39:30 2022] GET /identity/v3/roles/8427479f632a4da3949971f58b438c7f => generated 223 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:30.245153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd1a34-0aca-4a95-95f6-ea444f3830e1 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:30.346728 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd1a34-0aca-4a95-95f6-ea444f3830e1 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:30.381514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf56863-cc39-48d9-a372-f4212131a1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:30.382396 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf56863-cc39-48d9-a372-f4212131a1ae None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:30.382682 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf56863-cc39-48d9-a372-f4212131a1ae None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:30.382959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf56863-cc39-48d9-a372-f4212131a1ae None admin] PATH_INFO: `/v3/users/a63ea4c1aa66488fa15e7c2e1904cdc3` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:30.384189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf56863-cc39-48d9-a372-f4212131a1ae None admin] RBAC: Authorizing `identity:get_user(user_id=a63ea4c1aa66488fa15e7c2e1904cdc3)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:30.385624 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf56863-cc39-48d9-a372-f4212131a1ae None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:30.387278 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 562/1127] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:39:30 2022] GET /identity/v3/users/a63ea4c1aa66488fa15e7c2e1904cdc3 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:30.394898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e620b9-c6ee-4666-abd0-d0ae3e6c5d46 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:30.482921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:30.499415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e620b9-c6ee-4666-abd0-d0ae3e6c5d46 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:30.533931 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768d4b4-4789-4063-9120-7b7a35d2f6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:30.534582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768d4b4-4789-4063-9120-7b7a35d2f6c1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:30.534866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768d4b4-4789-4063-9120-7b7a35d2f6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:30.535144 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768d4b4-4789-4063-9120-7b7a35d2f6c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:30.604356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768d4b4-4789-4063-9120-7b7a35d2f6c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:30.605814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768d4b4-4789-4063-9120-7b7a35d2f6c1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:30.690697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:30.712835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 566/1128] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:39:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:39:30.718811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf56863-cc39-48d9-a372-f4212131a1ae None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:30.747494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:30.832911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf56863-cc39-48d9-a372-f4212131a1ae None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:30.870898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1e00a-4684-452a-a480-653e4872de42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:30.871789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1e00a-4684-452a-a480-653e4872de42 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:30.872112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1e00a-4684-452a-a480-653e4872de42 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:30.872398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1e00a-4684-452a-a480-653e4872de42 None admin] PATH_INFO: `/v3/projects/45139be5b6bb456e8323152cec5e3744` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:30.873680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1e00a-4684-452a-a480-653e4872de42 None admin] RBAC: Authorizing `identity:get_project(project_id=45139be5b6bb456e8323152cec5e3744)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:30.875717 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1e00a-4684-452a-a480-653e4872de42 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:30.877857 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 563/1129] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:39:30 2022] GET /identity/v3/projects/45139be5b6bb456e8323152cec5e3744 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:30.885220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768d4b4-4789-4063-9120-7b7a35d2f6c1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:30.994033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768d4b4-4789-4063-9120-7b7a35d2f6c1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:31.028742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefb1c3-9962-46e3-846a-5bb69ce2b13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:31.029318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefb1c3-9962-46e3-846a-5bb69ce2b13c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:31.029603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefb1c3-9962-46e3-846a-5bb69ce2b13c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:31.029882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefb1c3-9962-46e3-846a-5bb69ce2b13c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:31.030714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcefb1c3-9962-46e3-846a-5bb69ce2b13c None admin] RBAC: Adding query filter params (role.id=8427479f632a4da3949971f58b438c7f, scope.project.id=45139be5b6bb456e8323152cec5e3744, user.id=a63ea4c1aa66488fa15e7c2e1904cdc3) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:31.031092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcefb1c3-9962-46e3-846a-5bb69ce2b13c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:31.032590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcefb1c3-9962-46e3-846a-5bb69ce2b13c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:31.038567 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 567/1130] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:39:30 2022] GET /identity/v3/role_assignments?user.id=a63ea4c1aa66488fa15e7c2e1904cdc3&scope.project.id=45139be5b6bb456e8323152cec5e3744&role.id=8427479f632a4da3949971f58b438c7f => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:33.158910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fbe4b-cbc1-46a1-9a75-4fb76c7f7bab None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:33.159465 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fbe4b-cbc1-46a1-9a75-4fb76c7f7bab None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:33.159465 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fbe4b-cbc1-46a1-9a75-4fb76c7f7bab None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:33.160861 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 564/1131] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:39:33.166532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987dcb1-73fa-4448-b1cd-e81c1e3fe77f None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:33.166942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987dcb1-73fa-4448-b1cd-e81c1e3fe77f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:33.167214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987dcb1-73fa-4448-b1cd-e81c1e3fe77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:33.185058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-9987dcb1-73fa-4448-b1cd-e81c1e3fe77f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:33.186270 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9987dcb1-73fa-4448-b1cd-e81c1e3fe77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:33.232876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:33.260857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 568/1132] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:39:33.284436 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fbe4b-cbc1-46a1-9a75-4fb76c7f7bab None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:33.381314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:33.393144 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fbe4b-cbc1-46a1-9a75-4fb76c7f7bab None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:33.450551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d5696-27f2-4e2b-91c4-a20606cd437d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:33.451865 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d5696-27f2-4e2b-91c4-a20606cd437d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:33.452386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d5696-27f2-4e2b-91c4-a20606cd437d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:33.452882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d5696-27f2-4e2b-91c4-a20606cd437d None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:33.455452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d5696-27f2-4e2b-91c4-a20606cd437d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:33.458062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d5696-27f2-4e2b-91c4-a20606cd437d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:33.461198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 565/1133] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:33.467467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987dcb1-73fa-4448-b1cd-e81c1e3fe77f None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:33.567886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987dcb1-73fa-4448-b1cd-e81c1e3fe77f None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:33.604994 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd1097-e3c8-455e-86db-201c59691382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:33.605674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd1097-e3c8-455e-86db-201c59691382 None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:33.605955 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd1097-e3c8-455e-86db-201c59691382 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:33.606234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd1097-e3c8-455e-86db-201c59691382 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:33.606987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd1097-e3c8-455e-86db-201c59691382 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:33.608517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd1097-e3c8-455e-86db-201c59691382 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:33.636730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 569/1134] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Jul 14 17:39:33 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:39:35.781738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85188a-9155-421f-9871-39a6251e0d48 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:35.782326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85188a-9155-421f-9871-39a6251e0d48 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:35.782326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df85188a-9155-421f-9871-39a6251e0d48 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:35.783819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 566/1135] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:39:35.791912 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43adeda-a212-4535-9ad5-dab75a8ed718 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:35.792458 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43adeda-a212-4535-9ad5-dab75a8ed718 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:35.792580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43adeda-a212-4535-9ad5-dab75a8ed718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:35.810820 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-a43adeda-a212-4535-9ad5-dab75a8ed718 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:35.812079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a43adeda-a212-4535-9ad5-dab75a8ed718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:35.840127 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:35.888563 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 570/1136] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:35.914049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85188a-9155-421f-9871-39a6251e0d48 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:36.021629 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df85188a-9155-421f-9871-39a6251e0d48 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:36.059697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65952d7a-5a0d-4747-aa1c-ebc95d04da70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:36.060436 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65952d7a-5a0d-4747-aa1c-ebc95d04da70 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:36.060723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65952d7a-5a0d-4747-aa1c-ebc95d04da70 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:36.061001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65952d7a-5a0d-4747-aa1c-ebc95d04da70 None admin] PATH_INFO: `/v3/domains/default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:36.062204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65952d7a-5a0d-4747-aa1c-ebc95d04da70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:36.063725 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65952d7a-5a0d-4747-aa1c-ebc95d04da70 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:36.065430 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 567/1137] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:39:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:36.075330 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43adeda-a212-4535-9ad5-dab75a8ed718 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:36.261262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43adeda-a212-4535-9ad5-dab75a8ed718 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:36.298775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd5eb6-0288-4f07-8c9d-dc0e5094ca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:36.299458 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd5eb6-0288-4f07-8c9d-dc0e5094ca7d None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:36.299745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd5eb6-0288-4f07-8c9d-dc0e5094ca7d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:36.300054 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd5eb6-0288-4f07-8c9d-dc0e5094ca7d None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:36.300782 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd5eb6-0288-4f07-8c9d-dc0e5094ca7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:36.302244 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd5eb6-0288-4f07-8c9d-dc0e5094ca7d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:36.327433 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 571/1138] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:39:36 2022] POST /identity/v3/users => generated 286 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:39:38.490455 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfce8fe-6166-451e-ae72-f880f3d78d7b None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:38.491050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfce8fe-6166-451e-ae72-f880f3d78d7b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:38.491050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfce8fe-6166-451e-ae72-f880f3d78d7b None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:38.492663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 568/1139] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:38.498665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abb842-2720-44e0-84fe-5af0d3cf6212 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:38.499203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abb842-2720-44e0-84fe-5af0d3cf6212 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:38.499349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abb842-2720-44e0-84fe-5af0d3cf6212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:38.517529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-44abb842-2720-44e0-84fe-5af0d3cf6212 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:38.518745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-44abb842-2720-44e0-84fe-5af0d3cf6212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:38.531477 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:38.593448 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 572/1140] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:38.618303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfce8fe-6166-451e-ae72-f880f3d78d7b None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:38.653621 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:38.731851 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfce8fe-6166-451e-ae72-f880f3d78d7b None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:38.769719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517e2fe-0ac7-43fb-a033-10260fe7415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:38.770450 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517e2fe-0ac7-43fb-a033-10260fe7415e None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:38.770736 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517e2fe-0ac7-43fb-a033-10260fe7415e None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:38.771015 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e517e2fe-0ac7-43fb-a033-10260fe7415e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:38.782136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e517e2fe-0ac7-43fb-a033-10260fe7415e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-e517e2fe-0ac7-43fb-a033-10260fe7415e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:39:38.789985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:38.791808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:38.793863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:38.793863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:38.793863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:38.793863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:38.793863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:39:38.793863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:39:38.793863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:39:38.793863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:39:38.793863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:39:38.793863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:39:38.793863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:39:38.793863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:38.793863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 569/1141] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:39:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:39:38.799657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44abb842-2720-44e0-84fe-5af0d3cf6212 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:38.907821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44abb842-2720-44e0-84fe-5af0d3cf6212 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:38.947418 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a96fc-e6c3-4e54-bca6-0ddabf56551c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:38.948189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a96fc-e6c3-4e54-bca6-0ddabf56551c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:38.948550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a96fc-e6c3-4e54-bca6-0ddabf56551c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:38.948873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a96fc-e6c3-4e54-bca6-0ddabf56551c None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:38.949702 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a96fc-e6c3-4e54-bca6-0ddabf56551c None admin] RBAC: Adding query filter params (name=admin) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:38.950060 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a96fc-e6c3-4e54-bca6-0ddabf56551c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:38.951511 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a96fc-e6c3-4e54-bca6-0ddabf56551c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:38.961823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 573/1142] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:39:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:38.968319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517e2fe-0ac7-43fb-a033-10260fe7415e None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:39.077582 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:39.088654 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e517e2fe-0ac7-43fb-a033-10260fe7415e None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:39.145635 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcefb4b-e5e4-418a-a390-327d3dc9ae71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:39.147105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcefb4b-e5e4-418a-a390-327d3dc9ae71 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:39.147572 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcefb4b-e5e4-418a-a390-327d3dc9ae71 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:39.148072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcefb4b-e5e4-418a-a390-327d3dc9ae71 None admin] PATH_INFO: `/v3/users/848a6861acb14f29900e49fb12dbfdb1` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:39.171751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcefb4b-e5e4-418a-a390-327d3dc9ae71 None admin] RBAC: Authorizing `identity:get_user(user_id=848a6861acb14f29900e49fb12dbfdb1)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:39.174626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcefb4b-e5e4-418a-a390-327d3dc9ae71 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:39.177760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 570/1143] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:39:38 2022] GET /identity/v3/users/848a6861acb14f29900e49fb12dbfdb1 => generated 286 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:39.184624 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a96fc-e6c3-4e54-bca6-0ddabf56551c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:39.205648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:39.305744 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a96fc-e6c3-4e54-bca6-0ddabf56551c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:39.341252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b1dd4-6e79-4afa-b895-8cf21e997141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:39.341821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b1dd4-6e79-4afa-b895-8cf21e997141 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:39.342108 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b1dd4-6e79-4afa-b895-8cf21e997141 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:39.342387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b1dd4-6e79-4afa-b895-8cf21e997141 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:39.415886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b1dd4-6e79-4afa-b895-8cf21e997141 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:39.417555 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b1dd4-6e79-4afa-b895-8cf21e997141 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:39.428681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:39.532814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 574/1144] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:39:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:39:39.539518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcefb4b-e5e4-418a-a390-327d3dc9ae71 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:39.686662 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcefb4b-e5e4-418a-a390-327d3dc9ae71 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:39.725800 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ead059c-b607-42d2-9290-6a8215603d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:39.726714 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead059c-b607-42d2-9290-6a8215603d99 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:39.727002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead059c-b607-42d2-9290-6a8215603d99 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:39.727278 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ead059c-b607-42d2-9290-6a8215603d99 None admin] PATH_INFO: `/v3/projects/3a80549f8c544887b2bb57505cdc28e7` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:39.728784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead059c-b607-42d2-9290-6a8215603d99 None admin] RBAC: Authorizing `identity:get_project(project_id=3a80549f8c544887b2bb57505cdc28e7)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:39.730272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ead059c-b607-42d2-9290-6a8215603d99 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:39.732194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 571/1145] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:39:39 2022] GET /identity/v3/projects/3a80549f8c544887b2bb57505cdc28e7 => generated 311 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:39.739578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b1dd4-6e79-4afa-b895-8cf21e997141 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:39.849664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b1dd4-6e79-4afa-b895-8cf21e997141 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:39.860532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:39.889353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a2fef8-c4c3-45e5-a2a6-340e3ba6230e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:39.890039 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2fef8-c4c3-45e5-a2a6-340e3ba6230e None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:39.890310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2fef8-c4c3-45e5-a2a6-340e3ba6230e None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:39.890589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2fef8-c4c3-45e5-a2a6-340e3ba6230e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:39.892141 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2fef8-c4c3-45e5-a2a6-340e3ba6230e None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=3a80549f8c544887b2bb57505cdc28e7, user.id=848a6861acb14f29900e49fb12dbfdb1) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:39.892460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2fef8-c4c3-45e5-a2a6-340e3ba6230e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:39.893938 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a2fef8-c4c3-45e5-a2a6-340e3ba6230e None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:39.904064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 575/1146] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:39:39 2022] GET /identity/v3/role_assignments?user.id=848a6861acb14f29900e49fb12dbfdb1&scope.project.id=3a80549f8c544887b2bb57505cdc28e7&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:42.014036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8248d5c-8ed9-401d-859f-fb19537b9c6a None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:42.014586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8248d5c-8ed9-401d-859f-fb19537b9c6a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:42.014586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8248d5c-8ed9-401d-859f-fb19537b9c6a None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:42.016047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 572/1147] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:39:42.023032 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb06d8e-8bb0-4408-8ce6-31d705856a73 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:42.023700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb06d8e-8bb0-4408-8ce6-31d705856a73 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:42.023700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb06d8e-8bb0-4408-8ce6-31d705856a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:42.041329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-fdb06d8e-8bb0-4408-8ce6-31d705856a73 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:42.042550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-fdb06d8e-8bb0-4408-8ce6-31d705856a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:42.120839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 576/1148] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:39:42.145418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8248d5c-8ed9-401d-859f-fb19537b9c6a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:42.166867 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:42.270974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8248d5c-8ed9-401d-859f-fb19537b9c6a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:42.310837 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069816a7-d5f5-4ed0-b6d5-dc0abe3471ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:42.311676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069816a7-d5f5-4ed0-b6d5-dc0abe3471ce None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:42.311993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069816a7-d5f5-4ed0-b6d5-dc0abe3471ce None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:42.312280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069816a7-d5f5-4ed0-b6d5-dc0abe3471ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:42.324819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069816a7-d5f5-4ed0-b6d5-dc0abe3471ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-069816a7-d5f5-4ed0-b6d5-dc0abe3471ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:39:42.333410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:42.334460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:42.335306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:42.335306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:42.335306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:42.335306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:42.335306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:39:42.335306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:39:42.335306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:39:42.335306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:39:42.335306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:39:42.335306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:39:42.335306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:39:42.335306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:42.335306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 573/1149] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:39:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:42.341311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb06d8e-8bb0-4408-8ce6-31d705856a73 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:42.454049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb06d8e-8bb0-4408-8ce6-31d705856a73 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:42.455815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:42.492162 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45f8ae-60b9-4738-82de-aa9452004e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:42.492857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45f8ae-60b9-4738-82de-aa9452004e2a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:42.493142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45f8ae-60b9-4738-82de-aa9452004e2a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:42.493419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b45f8ae-60b9-4738-82de-aa9452004e2a None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:42.494196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45f8ae-60b9-4738-82de-aa9452004e2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:42.494508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45f8ae-60b9-4738-82de-aa9452004e2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:42.495958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b45f8ae-60b9-4738-82de-aa9452004e2a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:42.506541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 577/1150] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:39:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:42.516144 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069816a7-d5f5-4ed0-b6d5-dc0abe3471ce None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:42.653270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:42.678456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069816a7-d5f5-4ed0-b6d5-dc0abe3471ce None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:42.715268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1cc0f-f781-4dd9-be40-702a7a8faf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:42.716181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1cc0f-f781-4dd9-be40-702a7a8faf52 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:42.716462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1cc0f-f781-4dd9-be40-702a7a8faf52 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:42.716737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e1cc0f-f781-4dd9-be40-702a7a8faf52 None admin] PATH_INFO: `/v3/users/848a6861acb14f29900e49fb12dbfdb1` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:42.717971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1cc0f-f781-4dd9-be40-702a7a8faf52 None admin] RBAC: Authorizing `identity:get_user(user_id=848a6861acb14f29900e49fb12dbfdb1)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:42.719951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e1cc0f-f781-4dd9-be40-702a7a8faf52 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:42.721774 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 574/1151] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:39:42 2022] GET /identity/v3/users/848a6861acb14f29900e49fb12dbfdb1 => generated 286 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:42.727895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45f8ae-60b9-4738-82de-aa9452004e2a None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:42.837501 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b45f8ae-60b9-4738-82de-aa9452004e2a None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:42.873214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe1c59d-3f2b-45b8-b8e7-946dc967a488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:42.874100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1c59d-3f2b-45b8-b8e7-946dc967a488 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:42.874384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1c59d-3f2b-45b8-b8e7-946dc967a488 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:42.874664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe1c59d-3f2b-45b8-b8e7-946dc967a488 None admin] PATH_INFO: `/v3/projects/3a80549f8c544887b2bb57505cdc28e7` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:42.876503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1c59d-3f2b-45b8-b8e7-946dc967a488 None admin] RBAC: Authorizing `identity:get_project(project_id=3a80549f8c544887b2bb57505cdc28e7)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:42.878214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe1c59d-3f2b-45b8-b8e7-946dc967a488 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:42.880022 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 578/1152] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:39:42 2022] GET /identity/v3/projects/3a80549f8c544887b2bb57505cdc28e7 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:42.886494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1cc0f-f781-4dd9-be40-702a7a8faf52 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:42.993852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e1cc0f-f781-4dd9-be40-702a7a8faf52 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:43.037699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d58889-d9aa-45dd-bfcd-604d197ccc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:43.038606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d58889-d9aa-45dd-bfcd-604d197ccc58 None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:43.038890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d58889-d9aa-45dd-bfcd-604d197ccc58 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:43.039166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d58889-d9aa-45dd-bfcd-604d197ccc58 None admin] PATH_INFO: `/v3/projects/3a80549f8c544887b2bb57505cdc28e7/users/848a6861acb14f29900e49fb12dbfdb1/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:43.041271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d58889-d9aa-45dd-bfcd-604d197ccc58 None admin] RBAC: Authorizing `identity:create_grant(project_id=3a80549f8c544887b2bb57505cdc28e7, user_id=848a6861acb14f29900e49fb12dbfdb1, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:43.042721 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d58889-d9aa-45dd-bfcd-604d197ccc58 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:43.057554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 575/1153] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:39:42 2022] PUT /identity/v3/projects/3a80549f8c544887b2bb57505cdc28e7/users/848a6861acb14f29900e49fb12dbfdb1/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:39:45.162105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afa028-aa34-42c3-88a7-18e8200fc866 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:45.162666 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afa028-aa34-42c3-88a7-18e8200fc866 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:45.162666 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22afa028-aa34-42c3-88a7-18e8200fc866 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:45.164033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 579/1154] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:39:45.169921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805c6b1-ef9d-468b-8a60-9a476346d65b None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:45.170440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805c6b1-ef9d-468b-8a60-9a476346d65b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:45.170440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805c6b1-ef9d-468b-8a60-9a476346d65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:45.182065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:45.187917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-0805c6b1-ef9d-468b-8a60-9a476346d65b None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:45.189980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0805c6b1-ef9d-468b-8a60-9a476346d65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:45.259338 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 576/1155] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:45.282673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afa028-aa34-42c3-88a7-18e8200fc866 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:45.326468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:45.425850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22afa028-aa34-42c3-88a7-18e8200fc866 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:45.469964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690619f-2b8d-48a1-940d-08c3ac4173f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:45.470763 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690619f-2b8d-48a1-940d-08c3ac4173f5 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:45.471045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690619f-2b8d-48a1-940d-08c3ac4173f5 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:45.471320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e690619f-2b8d-48a1-940d-08c3ac4173f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:45.482765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e690619f-2b8d-48a1-940d-08c3ac4173f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-e690619f-2b8d-48a1-940d-08c3ac4173f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:39:45.491003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:45.492208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:45.493187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:45.493187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:45.493187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:45.493187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:45.493187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:39:45.493187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:39:45.493187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:39:45.493187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:39:45.493187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:39:45.493187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:39:45.493187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:39:45.493187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:45.493187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 580/1156] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:39:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:39:45.498728 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805c6b1-ef9d-468b-8a60-9a476346d65b None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:45.610653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805c6b1-ef9d-468b-8a60-9a476346d65b None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:45.659231 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f1acc-dedb-43ba-b03b-66b1f81b5f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:45.659984 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f1acc-dedb-43ba-b03b-66b1f81b5f47 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:45.660299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f1acc-dedb-43ba-b03b-66b1f81b5f47 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:45.660601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f1acc-dedb-43ba-b03b-66b1f81b5f47 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:45.661461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f1acc-dedb-43ba-b03b-66b1f81b5f47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:45.661789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f1acc-dedb-43ba-b03b-66b1f81b5f47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:45.663324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f1acc-dedb-43ba-b03b-66b1f81b5f47 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:45.666295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:45.674950 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 577/1157] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:39:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:45.683566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690619f-2b8d-48a1-940d-08c3ac4173f5 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:45.799743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:45.811087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e690619f-2b8d-48a1-940d-08c3ac4173f5 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:45.852887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45385f76-f23a-4405-a45a-964c7111f4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:45.853789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45385f76-f23a-4405-a45a-964c7111f4b4 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:45.854075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45385f76-f23a-4405-a45a-964c7111f4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:45.854356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45385f76-f23a-4405-a45a-964c7111f4b4 None admin] PATH_INFO: `/v3/users/848a6861acb14f29900e49fb12dbfdb1` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:45.855720 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45385f76-f23a-4405-a45a-964c7111f4b4 None admin] RBAC: Authorizing `identity:get_user(user_id=848a6861acb14f29900e49fb12dbfdb1)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:45.857256 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45385f76-f23a-4405-a45a-964c7111f4b4 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:45.859059 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 581/1158] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:39:45 2022] GET /identity/v3/users/848a6861acb14f29900e49fb12dbfdb1 => generated 286 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:45.865396 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f1acc-dedb-43ba-b03b-66b1f81b5f47 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:45.983778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f1acc-dedb-43ba-b03b-66b1f81b5f47 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:46.021882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82047b-0540-4afb-9e88-e1aaececdd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:46.022554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82047b-0540-4afb-9e88-e1aaececdd3b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:46.022840 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82047b-0540-4afb-9e88-e1aaececdd3b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:46.023119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d82047b-0540-4afb-9e88-e1aaececdd3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:46.094003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:46.100860 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82047b-0540-4afb-9e88-e1aaececdd3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:46.102890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d82047b-0540-4afb-9e88-e1aaececdd3b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:46.213619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 578/1159] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:39:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:39:46.220498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45385f76-f23a-4405-a45a-964c7111f4b4 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:46.327745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45385f76-f23a-4405-a45a-964c7111f4b4 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:46.364585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320049c2-ac36-408f-9383-0cc406bf192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:46.365453 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320049c2-ac36-408f-9383-0cc406bf192f None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:46.365738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320049c2-ac36-408f-9383-0cc406bf192f None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:46.366024 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320049c2-ac36-408f-9383-0cc406bf192f None admin] PATH_INFO: `/v3/projects/3a80549f8c544887b2bb57505cdc28e7` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:46.367288 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320049c2-ac36-408f-9383-0cc406bf192f None admin] RBAC: Authorizing `identity:get_project(project_id=3a80549f8c544887b2bb57505cdc28e7)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:46.368788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320049c2-ac36-408f-9383-0cc406bf192f None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:46.370570 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 582/1160] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:39:46 2022] GET /identity/v3/projects/3a80549f8c544887b2bb57505cdc28e7 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:46.377741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82047b-0540-4afb-9e88-e1aaececdd3b None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:46.484828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:46.492302 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d82047b-0540-4afb-9e88-e1aaececdd3b None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:46.551329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb4d2e-8183-46bd-9d20-c550a50cc320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:46.552488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb4d2e-8183-46bd-9d20-c550a50cc320 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:46.553006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb4d2e-8183-46bd-9d20-c550a50cc320 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:46.553511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb4d2e-8183-46bd-9d20-c550a50cc320 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:46.555148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb4d2e-8183-46bd-9d20-c550a50cc320 None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=3a80549f8c544887b2bb57505cdc28e7, user.id=848a6861acb14f29900e49fb12dbfdb1) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:46.555716 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb4d2e-8183-46bd-9d20-c550a50cc320 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:46.558236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb4d2e-8183-46bd-9d20-c550a50cc320 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:46.568842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 579/1161] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:39:46 2022] GET /identity/v3/role_assignments?user.id=848a6861acb14f29900e49fb12dbfdb1&scope.project.id=3a80549f8c544887b2bb57505cdc28e7&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:48.655718 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142bfb0-7c2f-4f11-b78c-2044f936a9f7 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:48.656342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142bfb0-7c2f-4f11-b78c-2044f936a9f7 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:48.656342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142bfb0-7c2f-4f11-b78c-2044f936a9f7 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:48.657813 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 583/1162] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:48.664026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd099a0-75fd-4f5d-8214-bc3fc4873ee7 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:48.664571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd099a0-75fd-4f5d-8214-bc3fc4873ee7 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:48.664571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd099a0-75fd-4f5d-8214-bc3fc4873ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:48.682379 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-9bd099a0-75fd-4f5d-8214-bc3fc4873ee7 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:48.683612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9bd099a0-75fd-4f5d-8214-bc3fc4873ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:48.758855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 580/1163] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:48.782250 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142bfb0-7c2f-4f11-b78c-2044f936a9f7 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:48.815653 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:48.903338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142bfb0-7c2f-4f11-b78c-2044f936a9f7 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:48.944363 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a84e56-0694-415a-b63b-3ef00fee614d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:48.944981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a84e56-0694-415a-b63b-3ef00fee614d None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:48.945317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a84e56-0694-415a-b63b-3ef00fee614d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:48.945650 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a84e56-0694-415a-b63b-3ef00fee614d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:49.028690 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a84e56-0694-415a-b63b-3ef00fee614d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:49.030229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a84e56-0694-415a-b63b-3ef00fee614d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:49.062946 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:49.147692 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 584/1164] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:39:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 17:39:49.153976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd099a0-75fd-4f5d-8214-bc3fc4873ee7 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:49.250712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:49.260678 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd099a0-75fd-4f5d-8214-bc3fc4873ee7 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:49.299379 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a93d9-4168-4837-b2f5-9eeff6f35b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:49.300295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a93d9-4168-4837-b2f5-9eeff6f35b89 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:49.300652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a93d9-4168-4837-b2f5-9eeff6f35b89 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:49.300976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a93d9-4168-4837-b2f5-9eeff6f35b89 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:49.308200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a93d9-4168-4837-b2f5-9eeff6f35b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:49.309846 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a93d9-4168-4837-b2f5-9eeff6f35b89 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-f40a93d9-4168-4837-b2f5-9eeff6f35b89 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:49.324451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:49.326469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:39:49.328697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 581/1165] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Jul 14 17:39:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:49.334697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a84e56-0694-415a-b63b-3ef00fee614d None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:49.438932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a84e56-0694-415a-b63b-3ef00fee614d None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:49.466826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:49.476186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b798cb75-1e15-4eab-85fe-bd8f8a15fbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:49.476871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798cb75-1e15-4eab-85fe-bd8f8a15fbdf None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:49.477154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798cb75-1e15-4eab-85fe-bd8f8a15fbdf None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:49.477432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b798cb75-1e15-4eab-85fe-bd8f8a15fbdf None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:49.478172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798cb75-1e15-4eab-85fe-bd8f8a15fbdf None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:49.478485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798cb75-1e15-4eab-85fe-bd8f8a15fbdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:49.479865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b798cb75-1e15-4eab-85fe-bd8f8a15fbdf None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:49.488533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 585/1166] 10.222.0.64 () {56 vars in 1115 bytes} [Thu Jul 14 17:39:49 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:39:51.613772 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd3cfe-2f1d-4832-98b1-7c448f0e761d None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:51.614329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd3cfe-2f1d-4832-98b1-7c448f0e761d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:51.614494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcd3cfe-2f1d-4832-98b1-7c448f0e761d None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:51.615876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 582/1167] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:39:51.621630 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9d2c0-ebef-4924-9659-977380d7949a None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:51.622170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9d2c0-ebef-4924-9659-977380d7949a None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:51.622292 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9d2c0-ebef-4924-9659-977380d7949a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:51.649534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4da9d2c0-ebef-4924-9659-977380d7949a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:51.650857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4da9d2c0-ebef-4924-9659-977380d7949a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:51.734978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 586/1168] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:39:51.761535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd3cfe-2f1d-4832-98b1-7c448f0e761d None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:51.869160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcd3cfe-2f1d-4832-98b1-7c448f0e761d None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:51.904976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba9721-2bb1-4b0f-8b51-2492c0f5db89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:51.905721 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba9721-2bb1-4b0f-8b51-2492c0f5db89 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:51.906048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba9721-2bb1-4b0f-8b51-2492c0f5db89 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:51.906369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba9721-2bb1-4b0f-8b51-2492c0f5db89 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:51.907190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba9721-2bb1-4b0f-8b51-2492c0f5db89 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:51.908665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba9721-2bb1-4b0f-8b51-2492c0f5db89 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:51.925705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 583/1169] 10.222.0.64 () {60 vars in 1138 bytes} [Thu Jul 14 17:39:51 2022] POST /identity/v3/domains => generated 267 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:39:54.092831 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fefa9-24a0-4661-91de-71ff17dd4cb8 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:54.093529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fefa9-24a0-4661-91de-71ff17dd4cb8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:54.093529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fefa9-24a0-4661-91de-71ff17dd4cb8 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:54.094835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 587/1170] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:54.103758 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a823b-b374-4134-b063-3ba4aa4a8050 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:54.104329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a823b-b374-4134-b063-3ba4aa4a8050 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:54.104431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a823b-b374-4134-b063-3ba4aa4a8050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:54.123986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f03a823b-b374-4134-b063-3ba4aa4a8050 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:54.125243 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f03a823b-b374-4134-b063-3ba4aa4a8050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:54.133974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:54.207801 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 584/1171] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:54.246141 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fefa9-24a0-4661-91de-71ff17dd4cb8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:54.411534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fefa9-24a0-4661-91de-71ff17dd4cb8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:54.436333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:54.449547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b6815-f09b-47ff-a38b-64e70c388d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:54.450433 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b6815-f09b-47ff-a38b-64e70c388d43 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:54.450721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b6815-f09b-47ff-a38b-64e70c388d43 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:54.451001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b6815-f09b-47ff-a38b-64e70c388d43 None admin] PATH_INFO: `/v3/domains/f87b0d279a004ac7a98dadb26bae0eaa` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:54.463016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b6815-f09b-47ff-a38b-64e70c388d43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f87b0d279a004ac7a98dadb26bae0eaa)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:54.464542 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03b6815-f09b-47ff-a38b-64e70c388d43 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:54.466350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 588/1172] 10.222.0.64 () {56 vars in 1183 bytes} [Thu Jul 14 17:39:54 2022] GET /identity/v3/domains/f87b0d279a004ac7a98dadb26bae0eaa => generated 267 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:54.473323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a823b-b374-4134-b063-3ba4aa4a8050 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:54.584794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a823b-b374-4134-b063-3ba4aa4a8050 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:54.621960 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927079a-396b-4305-a1d9-ce990322bf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:54.622646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927079a-396b-4305-a1d9-ce990322bf74 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:54.622936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927079a-396b-4305-a1d9-ce990322bf74 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:54.623219 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927079a-396b-4305-a1d9-ce990322bf74 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:54.623975 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927079a-396b-4305-a1d9-ce990322bf74 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:54.625450 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927079a-396b-4305-a1d9-ce990322bf74 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:54.640921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:54.659626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 585/1173] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Jul 14 17:39:54 2022] POST /identity/v3/projects => generated 361 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:39:56.800936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b2cce-765c-489b-8fbf-7ddc8ed7baa8 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:56.801774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b2cce-765c-489b-8fbf-7ddc8ed7baa8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:56.801933 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b2cce-765c-489b-8fbf-7ddc8ed7baa8 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:56.803358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 589/1174] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:39:56.808980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dc79f-f555-4c5d-b8f7-dfabf3febb27 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:56.809493 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dc79f-f555-4c5d-b8f7-dfabf3febb27 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:56.809625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795dc79f-f555-4c5d-b8f7-dfabf3febb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:56.828163 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-795dc79f-f555-4c5d-b8f7-dfabf3febb27 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:56.829522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-795dc79f-f555-4c5d-b8f7-dfabf3febb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:56.900648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 586/1175] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:39:56.927188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b2cce-765c-489b-8fbf-7ddc8ed7baa8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:57.035126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b2cce-765c-489b-8fbf-7ddc8ed7baa8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:57.071911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6dc3b-ea82-447d-9ac3-74239585f4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:57.072718 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6dc3b-ea82-447d-9ac3-74239585f4b8 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:57.073000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6dc3b-ea82-447d-9ac3-74239585f4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:57.073270 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6dc3b-ea82-447d-9ac3-74239585f4b8 None admin] PATH_INFO: `/v3/domains/f87b0d279a004ac7a98dadb26bae0eaa` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:57.074511 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6dc3b-ea82-447d-9ac3-74239585f4b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f87b0d279a004ac7a98dadb26bae0eaa)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:57.075975 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6dc3b-ea82-447d-9ac3-74239585f4b8 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:57.077648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 590/1176] 10.222.0.64 () {56 vars in 1183 bytes} [Thu Jul 14 17:39:56 2022] GET /identity/v3/domains/f87b0d279a004ac7a98dadb26bae0eaa => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:57.086715 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795dc79f-f555-4c5d-b8f7-dfabf3febb27 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:57.192051 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795dc79f-f555-4c5d-b8f7-dfabf3febb27 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:57.217489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:57.230478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b75204-9f77-4b75-9d57-6418133c7fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:57.231218 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b75204-9f77-4b75-9d57-6418133c7fa5 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:57.231547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b75204-9f77-4b75-9d57-6418133c7fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:57.231866 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b75204-9f77-4b75-9d57-6418133c7fa5 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:57.232670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b75204-9f77-4b75-9d57-6418133c7fa5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:57.234157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b75204-9f77-4b75-9d57-6418133c7fa5 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:57.256775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 587/1177] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:39:57 2022] POST /identity/v3/users => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:39:59.386207 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8397042-72f4-4dd5-b657-5f0498846feb None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:59.386730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8397042-72f4-4dd5-b657-5f0498846feb None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:59.387061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8397042-72f4-4dd5-b657-5f0498846feb None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:59.389671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 591/1178] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:39:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:39:59.395552 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecae869-3d16-4796-a412-2eb809617047 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:59.396083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecae869-3d16-4796-a412-2eb809617047 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:59.396191 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecae869-3d16-4796-a412-2eb809617047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:59.414098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-3ecae869-3d16-4796-a412-2eb809617047 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:39:59.415365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3ecae869-3d16-4796-a412-2eb809617047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:39:59.488240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 588/1179] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:39:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:39:59.512453 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8397042-72f4-4dd5-b657-5f0498846feb None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:59.569261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:59.627594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8397042-72f4-4dd5-b657-5f0498846feb None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:59.663962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0c61e-294c-4186-a3c4-1c4cc4614f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:59.664685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0c61e-294c-4186-a3c4-1c4cc4614f3e None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:59.664969 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0c61e-294c-4186-a3c4-1c4cc4614f3e None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:59.665248 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0c61e-294c-4186-a3c4-1c4cc4614f3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:59.678055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0c61e-294c-4186-a3c4-1c4cc4614f3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-d6a0c61e-294c-4186-a3c4-1c4cc4614f3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:39:59.686076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:39:59.687657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:39:59.689296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:39:59.689296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:39:59.689296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:39:59.689296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:39:59.689296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:39:59.689296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:39:59.689296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:39:59.689296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:39:59.689296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:39:59.689296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:39:59.689296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:39:59.689296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:39:59.689296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 592/1180] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:39:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:39:59.695404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecae869-3d16-4796-a412-2eb809617047 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:59.817625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecae869-3d16-4796-a412-2eb809617047 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:39:59.831174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:39:59.860195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0413b-b66a-41f1-9106-64067a8ad661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:39:59.861423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0413b-b66a-41f1-9106-64067a8ad661 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:39:59.861704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0413b-b66a-41f1-9106-64067a8ad661 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:39:59.861982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0413b-b66a-41f1-9106-64067a8ad661 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:39:59.862790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0413b-b66a-41f1-9106-64067a8ad661 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:39:59.863101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0413b-b66a-41f1-9106-64067a8ad661 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:39:59.864605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0413b-b66a-41f1-9106-64067a8ad661 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:39:59.876082 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 589/1181] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:39:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:39:59.883311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0c61e-294c-4186-a3c4-1c4cc4614f3e None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:39:59.993856 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0c61e-294c-4186-a3c4-1c4cc4614f3e None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:00.006605 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:40:00.039695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc8421-0617-4bae-aff6-f08f247a0aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:00.040614 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc8421-0617-4bae-aff6-f08f247a0aa0 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:00.040894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc8421-0617-4bae-aff6-f08f247a0aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:00.041166 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc8421-0617-4bae-aff6-f08f247a0aa0 None admin] PATH_INFO: `/v3/users/0444307493e74520b95dfd34a9fd4843` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:00.057298 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc8421-0617-4bae-aff6-f08f247a0aa0 None admin] RBAC: Authorizing `identity:get_user(user_id=0444307493e74520b95dfd34a9fd4843)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:40:00.058779 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc8421-0617-4bae-aff6-f08f247a0aa0 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:40:00.060617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 593/1182] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:39:59 2022] GET /identity/v3/users/0444307493e74520b95dfd34a9fd4843 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:40:00.066780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0413b-b66a-41f1-9106-64067a8ad661 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:40:00.176597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0413b-b66a-41f1-9106-64067a8ad661 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:00.211398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d046e9-d8ed-4951-bcdf-872737f4031a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:00.212092 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d046e9-d8ed-4951-bcdf-872737f4031a None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:00.212383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d046e9-d8ed-4951-bcdf-872737f4031a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:00.212666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d046e9-d8ed-4951-bcdf-872737f4031a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:00.257451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:40:00.285797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d046e9-d8ed-4951-bcdf-872737f4031a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:40:00.287267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d046e9-d8ed-4951-bcdf-872737f4031a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:40:00.386679 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 590/1183] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:40:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:40:00.394116 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc8421-0617-4bae-aff6-f08f247a0aa0 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:40:00.496082 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc8421-0617-4bae-aff6-f08f247a0aa0 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:00.531996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ef15f-fc58-4a33-98fa-daaa273b9c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:00.533452 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ef15f-fc58-4a33-98fa-daaa273b9c29 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:00.533735 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ef15f-fc58-4a33-98fa-daaa273b9c29 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:00.534014 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ef15f-fc58-4a33-98fa-daaa273b9c29 None admin] PATH_INFO: `/v3/projects/d57b6e0a44f34744ae0852896a72dabc` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:00.535237 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ef15f-fc58-4a33-98fa-daaa273b9c29 None admin] RBAC: Authorizing `identity:get_project(project_id=d57b6e0a44f34744ae0852896a72dabc)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:40:00.536714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ef15f-fc58-4a33-98fa-daaa273b9c29 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:40:00.538455 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 594/1184] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:40:00 2022] GET /identity/v3/projects/d57b6e0a44f34744ae0852896a72dabc => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:40:00.544913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d046e9-d8ed-4951-bcdf-872737f4031a None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:40:00.616542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:40:00.646688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d046e9-d8ed-4951-bcdf-872737f4031a None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:00.681644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a819c79-552d-487e-a7d8-a845b63b76b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:00.682310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a819c79-552d-487e-a7d8-a845b63b76b3 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:00.682593 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a819c79-552d-487e-a7d8-a845b63b76b3 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:00.682870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a819c79-552d-487e-a7d8-a845b63b76b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:00.683705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a819c79-552d-487e-a7d8-a845b63b76b3 None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=d57b6e0a44f34744ae0852896a72dabc, user.id=0444307493e74520b95dfd34a9fd4843) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:40:00.684048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a819c79-552d-487e-a7d8-a845b63b76b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:40:00.685477 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a819c79-552d-487e-a7d8-a845b63b76b3 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:40:00.691215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 591/1185] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:40:00 2022] GET /identity/v3/role_assignments?user.id=0444307493e74520b95dfd34a9fd4843&scope.project.id=d57b6e0a44f34744ae0852896a72dabc&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:40:02.742328 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702520ce-c0de-4877-b3ad-49db046e7279 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:02.742928 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702520ce-c0de-4877-b3ad-49db046e7279 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:02.742928 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702520ce-c0de-4877-b3ad-49db046e7279 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:02.744319 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 595/1186] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:40:02.751412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2977ccf-4db4-4c5b-90e7-a51d855fe2b1 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:02.752004 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2977ccf-4db4-4c5b-90e7-a51d855fe2b1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:02.752466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2977ccf-4db4-4c5b-90e7-a51d855fe2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:02.771637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-e2977ccf-4db4-4c5b-90e7-a51d855fe2b1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:40:02.772901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e2977ccf-4db4-4c5b-90e7-a51d855fe2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:40:02.847746 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 592/1187] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:40:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:40:02.871287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702520ce-c0de-4877-b3ad-49db046e7279 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:40:02.911298 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:40:02.985651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702520ce-c0de-4877-b3ad-49db046e7279 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:03.025195 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e358cbb-06f0-4949-8578-3a97a798b1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:03.025974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e358cbb-06f0-4949-8578-3a97a798b1c8 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:03.026304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e358cbb-06f0-4949-8578-3a97a798b1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:03.026632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e358cbb-06f0-4949-8578-3a97a798b1c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:03.039805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e358cbb-06f0-4949-8578-3a97a798b1c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-2e358cbb-06f0-4949-8578-3a97a798b1c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:40:03.047894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:40:03.049991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:40:03.051154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:40:03.051154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:40:03.051154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:40:03.051154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:40:03.051154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:40:03.051154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:40:03.051154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:40:03.051154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:40:03.051154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:40:03.051154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:40:03.051154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:40:03.051154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:40:03.051154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 596/1188] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:40:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:40:03.056583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2977ccf-4db4-4c5b-90e7-a51d855fe2b1 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:40:03.134598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:40:03.174185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2977ccf-4db4-4c5b-90e7-a51d855fe2b1 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:03.214110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f94f80-47ac-474f-9bdb-790579de6f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:03.215195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f94f80-47ac-474f-9bdb-790579de6f4f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:03.215674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f94f80-47ac-474f-9bdb-790579de6f4f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:03.216172 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f94f80-47ac-474f-9bdb-790579de6f4f None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:03.217464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f94f80-47ac-474f-9bdb-790579de6f4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:40:03.217787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f94f80-47ac-474f-9bdb-790579de6f4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:40:03.219419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f94f80-47ac-474f-9bdb-790579de6f4f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:40:03.234543 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 593/1189] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:40:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:40:03.240938 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e358cbb-06f0-4949-8578-3a97a798b1c8 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:40:03.346262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:40:03.352920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e358cbb-06f0-4949-8578-3a97a798b1c8 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:03.390618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895146f8-e06d-412a-9882-c29d376624c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:03.391500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895146f8-e06d-412a-9882-c29d376624c1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:03.391786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895146f8-e06d-412a-9882-c29d376624c1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:03.392102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895146f8-e06d-412a-9882-c29d376624c1 None admin] PATH_INFO: `/v3/users/0444307493e74520b95dfd34a9fd4843` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:03.393346 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895146f8-e06d-412a-9882-c29d376624c1 None admin] RBAC: Authorizing `identity:get_user(user_id=0444307493e74520b95dfd34a9fd4843)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:40:03.394803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895146f8-e06d-412a-9882-c29d376624c1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:40:03.396521 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 597/1190] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:40:03 2022] GET /identity/v3/users/0444307493e74520b95dfd34a9fd4843 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:40:03.403024 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f94f80-47ac-474f-9bdb-790579de6f4f None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:40:03.516969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f94f80-47ac-474f-9bdb-790579de6f4f None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:03.544021 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:40:03.553232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24adcc-2c6d-4f24-8d9e-0772fc601495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:03.554109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24adcc-2c6d-4f24-8d9e-0772fc601495 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:03.554390 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24adcc-2c6d-4f24-8d9e-0772fc601495 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:03.554666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24adcc-2c6d-4f24-8d9e-0772fc601495 None admin] PATH_INFO: `/v3/projects/d57b6e0a44f34744ae0852896a72dabc` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:03.555841 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24adcc-2c6d-4f24-8d9e-0772fc601495 None admin] RBAC: Authorizing `identity:get_project(project_id=d57b6e0a44f34744ae0852896a72dabc)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:40:03.557324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24adcc-2c6d-4f24-8d9e-0772fc601495 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:40:03.559030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 594/1191] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:40:03 2022] GET /identity/v3/projects/d57b6e0a44f34744ae0852896a72dabc => generated 361 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:40:03.566173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895146f8-e06d-412a-9882-c29d376624c1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:40:03.683907 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895146f8-e06d-412a-9882-c29d376624c1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:03.716423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a6dce-e337-4b27-bfac-9d2c6e5b047d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:03.717304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a6dce-e337-4b27-bfac-9d2c6e5b047d None admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:03.717587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a6dce-e337-4b27-bfac-9d2c6e5b047d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:03.717859 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a6dce-e337-4b27-bfac-9d2c6e5b047d None admin] PATH_INFO: `/v3/projects/d57b6e0a44f34744ae0852896a72dabc/users/0444307493e74520b95dfd34a9fd4843/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:03.719846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a6dce-e337-4b27-bfac-9d2c6e5b047d None admin] RBAC: Authorizing `identity:create_grant(project_id=d57b6e0a44f34744ae0852896a72dabc, user_id=0444307493e74520b95dfd34a9fd4843, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:40:03.721883 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a6dce-e337-4b27-bfac-9d2c6e5b047d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:40:03.739243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 598/1192] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:40:03 2022] PUT /identity/v3/projects/d57b6e0a44f34744ae0852896a72dabc/users/0444307493e74520b95dfd34a9fd4843/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:40:05.881103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a1e3e-5e74-47c3-b6c0-803e5d21e19e None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:05.881675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a1e3e-5e74-47c3-b6c0-803e5d21e19e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:05.881675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a1e3e-5e74-47c3-b6c0-803e5d21e19e None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:05.883162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 595/1193] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:40:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:40:05.888864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122152bc-ae57-4dc1-b332-a11d4a55de12 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:05.889385 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122152bc-ae57-4dc1-b332-a11d4a55de12 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:05.889512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122152bc-ae57-4dc1-b332-a11d4a55de12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:05.906390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-122152bc-ae57-4dc1-b332-a11d4a55de12 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:40:05.907625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-122152bc-ae57-4dc1-b332-a11d4a55de12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:40:05.926966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:40:05.980321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 599/1194] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:40:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:40:06.004801 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a1e3e-5e74-47c3-b6c0-803e5d21e19e None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:40:06.113904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a1e3e-5e74-47c3-b6c0-803e5d21e19e None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:06.149588 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac2f7f-96f7-43a7-8044-f3ba1765fbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:06.150319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac2f7f-96f7-43a7-8044-f3ba1765fbb9 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:06.150639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac2f7f-96f7-43a7-8044-f3ba1765fbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:06.150921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cac2f7f-96f7-43a7-8044-f3ba1765fbb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:06.163285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cac2f7f-96f7-43a7-8044-f3ba1765fbb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-2cac2f7f-96f7-43a7-8044-f3ba1765fbb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:40:06.171052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:40:06.172979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:40:06.175539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:40:06.175539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:40:06.175539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:40:06.175539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:40:06.175539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:40:06.175539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:40:06.175539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:40:06.175539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:40:06.175539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:40:06.175539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:40:06.175539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:40:06.175539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:40:06.175539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 596/1195] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:40:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:40:06.180628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122152bc-ae57-4dc1-b332-a11d4a55de12 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:40:06.282926 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122152bc-ae57-4dc1-b332-a11d4a55de12 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:06.320469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671b6cf-4b1e-4ffc-9cdc-d2a2634c06ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:06.321230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671b6cf-4b1e-4ffc-9cdc-d2a2634c06ae None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:06.321561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671b6cf-4b1e-4ffc-9cdc-d2a2634c06ae None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:06.321886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7671b6cf-4b1e-4ffc-9cdc-d2a2634c06ae None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:06.322700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671b6cf-4b1e-4ffc-9cdc-d2a2634c06ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:40:06.323056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671b6cf-4b1e-4ffc-9cdc-d2a2634c06ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:40:06.324510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7671b6cf-4b1e-4ffc-9cdc-d2a2634c06ae None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:40:06.333392 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 600/1196] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:40:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:40:06.339417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac2f7f-96f7-43a7-8044-f3ba1765fbb9 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:40:06.398386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:40:06.444674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cac2f7f-96f7-43a7-8044-f3ba1765fbb9 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:06.481040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309633a-1ee3-4fe2-9e21-c0e4c7e2f8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:06.481940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309633a-1ee3-4fe2-9e21-c0e4c7e2f8e7 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:06.482220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309633a-1ee3-4fe2-9e21-c0e4c7e2f8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:06.482495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6309633a-1ee3-4fe2-9e21-c0e4c7e2f8e7 None admin] PATH_INFO: `/v3/users/0444307493e74520b95dfd34a9fd4843` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:06.483709 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309633a-1ee3-4fe2-9e21-c0e4c7e2f8e7 None admin] RBAC: Authorizing `identity:get_user(user_id=0444307493e74520b95dfd34a9fd4843)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:40:06.485188 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6309633a-1ee3-4fe2-9e21-c0e4c7e2f8e7 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:40:06.487369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 597/1197] 10.222.0.64 () {56 vars in 1177 bytes} [Thu Jul 14 17:40:06 2022] GET /identity/v3/users/0444307493e74520b95dfd34a9fd4843 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:40:06.493338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671b6cf-4b1e-4ffc-9cdc-d2a2634c06ae None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:40:06.513103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:40:06.595937 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7671b6cf-4b1e-4ffc-9cdc-d2a2634c06ae None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:06.631849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356dd21-9735-49c4-96cd-46c7c53612ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:06.632529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356dd21-9735-49c4-96cd-46c7c53612ed None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:06.632812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356dd21-9735-49c4-96cd-46c7c53612ed None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:06.633085 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356dd21-9735-49c4-96cd-46c7c53612ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:06.703092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356dd21-9735-49c4-96cd-46c7c53612ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:40:06.704584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356dd21-9735-49c4-96cd-46c7c53612ed None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:40:06.722784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:40:06.812266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 601/1198] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:40:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:40:06.822105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309633a-1ee3-4fe2-9e21-c0e4c7e2f8e7 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:40:06.927692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6309633a-1ee3-4fe2-9e21-c0e4c7e2f8e7 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:06.960828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:40:06.978732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae48565-fe29-4b7b-afe2-d8acee749ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:06.980266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae48565-fe29-4b7b-afe2-d8acee749ec7 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:06.980741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae48565-fe29-4b7b-afe2-d8acee749ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:06.981181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae48565-fe29-4b7b-afe2-d8acee749ec7 None admin] PATH_INFO: `/v3/projects/d57b6e0a44f34744ae0852896a72dabc` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:06.983233 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae48565-fe29-4b7b-afe2-d8acee749ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=d57b6e0a44f34744ae0852896a72dabc)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:40:06.985949 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae48565-fe29-4b7b-afe2-d8acee749ec7 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:40:06.989536 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 598/1199] 10.222.0.64 () {56 vars in 1186 bytes} [Thu Jul 14 17:40:06 2022] GET /identity/v3/projects/d57b6e0a44f34744ae0852896a72dabc => generated 361 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:40:06.996015 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356dd21-9735-49c4-96cd-46c7c53612ed None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:40:07.100184 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356dd21-9735-49c4-96cd-46c7c53612ed None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:40:07.118543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:40:07.137745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43458f07-06fe-48ce-a696-3aab176dd212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:40:07.138410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43458f07-06fe-48ce-a696-3aab176dd212 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:40:07.138690 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43458f07-06fe-48ce-a696-3aab176dd212 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:40:07.138962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43458f07-06fe-48ce-a696-3aab176dd212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:40:07.139799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43458f07-06fe-48ce-a696-3aab176dd212 None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=d57b6e0a44f34744ae0852896a72dabc, user.id=0444307493e74520b95dfd34a9fd4843) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:40:07.140145 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43458f07-06fe-48ce-a696-3aab176dd212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:40:07.141580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43458f07-06fe-48ce-a696-3aab176dd212 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:40:07.148814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 602/1200] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:40:06 2022] GET /identity/v3/role_assignments?user.id=0444307493e74520b95dfd34a9fd4843&scope.project.id=d57b6e0a44f34744ae0852896a72dabc&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:40:16.971849 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=4 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:40:17.145671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=10 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:07.579041 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea2f80-348f-4f83-ba6f-df9816e6dd7a None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:07.580107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea2f80-348f-4f83-ba6f-df9816e6dd7a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:07.580107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea2f80-348f-4f83-ba6f-df9816e6dd7a None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:07.581561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 599/1201] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:41:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:41:07.588201 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d05f78-b4d5-4d12-9a17-35b0c1ab32d0 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:07.589161 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d05f78-b4d5-4d12-9a17-35b0c1ab32d0 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:07.589161 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d05f78-b4d5-4d12-9a17-35b0c1ab32d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:07.617962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-d8d05f78-b4d5-4d12-9a17-35b0c1ab32d0 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:41:07.619148 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d8d05f78-b4d5-4d12-9a17-35b0c1ab32d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:41:07.733342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 603/1202] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:41:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:41:07.765009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea2f80-348f-4f83-ba6f-df9816e6dd7a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:07.971977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea2f80-348f-4f83-ba6f-df9816e6dd7a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:08.031503 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9b7a8-a650-4373-b896-015997c56442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:08.033036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9b7a8-a650-4373-b896-015997c56442 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:08.033695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9b7a8-a650-4373-b896-015997c56442 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:08.034578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9b7a8-a650-4373-b896-015997c56442 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:08.057891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9b7a8-a650-4373-b896-015997c56442 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:08.060545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9b7a8-a650-4373-b896-015997c56442 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-0fb9b7a8-a650-4373-b896-015997c56442 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:41:08.100661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:08.102693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:08.104364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:41:08.104364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:41:08.104364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:08.104364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:08.104364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 600/1203] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:08.109967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d05f78-b4d5-4d12-9a17-35b0c1ab32d0 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:08.282993 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d05f78-b4d5-4d12-9a17-35b0c1ab32d0 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:08.336799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:08.345370 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79452e36-c617-47b0-a8ab-2fed2c6ffeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:08.346133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79452e36-c617-47b0-a8ab-2fed2c6ffeba None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:08.346525 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79452e36-c617-47b0-a8ab-2fed2c6ffeba None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:08.347141 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79452e36-c617-47b0-a8ab-2fed2c6ffeba None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:08.349280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79452e36-c617-47b0-a8ab-2fed2c6ffeba None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:08.349911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79452e36-c617-47b0-a8ab-2fed2c6ffeba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:08.352033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79452e36-c617-47b0-a8ab-2fed2c6ffeba None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:08.367206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 604/1204] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:08.375010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9b7a8-a650-4373-b896-015997c56442 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:08.499552 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:08.605421 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9b7a8-a650-4373-b896-015997c56442 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:08.683109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b225d593-da78-4a82-945c-db0cbf03f858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:08.684441 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225d593-da78-4a82-945c-db0cbf03f858 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:08.684966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225d593-da78-4a82-945c-db0cbf03f858 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:08.685309 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225d593-da78-4a82-945c-db0cbf03f858 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:08.686117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225d593-da78-4a82-945c-db0cbf03f858 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:08.687718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b225d593-da78-4a82-945c-db0cbf03f858 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:08.720035 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 601/1205] 10.222.0.64 () {60 vars in 1132 bytes} [Thu Jul 14 17:41:08 2022] POST /identity/v3/users => generated 251 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:41:11.321398 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7545555-08ff-47d9-a718-b2c344c5feb1 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:11.322212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7545555-08ff-47d9-a718-b2c344c5feb1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:11.322422 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7545555-08ff-47d9-a718-b2c344c5feb1 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:11.324269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 605/1206] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:41:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:41:11.332049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea08af9-ff34-4c22-9b5f-6233631dbd11 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:11.332970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea08af9-ff34-4c22-9b5f-6233631dbd11 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:11.332970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea08af9-ff34-4c22-9b5f-6233631dbd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:11.356966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-eea08af9-ff34-4c22-9b5f-6233631dbd11 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:41:11.358976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-eea08af9-ff34-4c22-9b5f-6233631dbd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:41:11.468315 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 602/1207] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:41:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:41:11.494361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7545555-08ff-47d9-a718-b2c344c5feb1 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:11.667780 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7545555-08ff-47d9-a718-b2c344c5feb1 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:11.735680 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acb1c3-62ed-4639-8e4e-7c7b2061916c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:11.736714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acb1c3-62ed-4639-8e4e-7c7b2061916c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:11.737135 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acb1c3-62ed-4639-8e4e-7c7b2061916c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:11.738600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49acb1c3-62ed-4639-8e4e-7c7b2061916c None admin] PATH_INFO: `/v3/roles/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:11.760221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49acb1c3-62ed-4639-8e4e-7c7b2061916c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-49acb1c3-62ed-4639-8e4e-7c7b2061916c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:41:11.772908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:11.782222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:11.783984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:11.783984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:11.783984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:11.783984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:11.783984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:41:11.783984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:41:11.783984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:41:11.783984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:41:11.783984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:41:11.783984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:41:11.783984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:41:11.783984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:11.783984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 606/1208] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:41:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:41:11.788667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea08af9-ff34-4c22-9b5f-6233631dbd11 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:11.843380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:11.929395 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea08af9-ff34-4c22-9b5f-6233631dbd11 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:11.978855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c468e98-b735-46a4-b1f8-ff90bb74bf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:11.979603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c468e98-b735-46a4-b1f8-ff90bb74bf9d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:11.979975 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c468e98-b735-46a4-b1f8-ff90bb74bf9d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:11.981121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c468e98-b735-46a4-b1f8-ff90bb74bf9d None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:11.982874 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c468e98-b735-46a4-b1f8-ff90bb74bf9d None admin] RBAC: Adding query filter params (name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:11.983308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c468e98-b735-46a4-b1f8-ff90bb74bf9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:11.984734 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c468e98-b735-46a4-b1f8-ff90bb74bf9d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:12.004270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 603/1209] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:41:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:12.010411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acb1c3-62ed-4639-8e4e-7c7b2061916c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:12.096126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:12.184263 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49acb1c3-62ed-4639-8e4e-7c7b2061916c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:12.244479 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3420440-57d0-4aed-9f6a-7c3e0014b33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:12.246210 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3420440-57d0-4aed-9f6a-7c3e0014b33c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:12.246929 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3420440-57d0-4aed-9f6a-7c3e0014b33c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:12.247608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3420440-57d0-4aed-9f6a-7c3e0014b33c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:12.269079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3420440-57d0-4aed-9f6a-7c3e0014b33c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:12.270431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3420440-57d0-4aed-9f6a-7c3e0014b33c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-f3420440-57d0-4aed-9f6a-7c3e0014b33c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:41:12.292106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:12.293337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:12.299350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:41:12.299350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:41:12.299350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:12.299350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:12.299350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 607/1210] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:41:12.299877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c468e98-b735-46a4-b1f8-ff90bb74bf9d None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:12.484355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:12.488352 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c468e98-b735-46a4-b1f8-ff90bb74bf9d None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:12.558839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6482128-e20b-4533-ba72-3f7d06ebe313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:12.560028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6482128-e20b-4533-ba72-3f7d06ebe313 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:12.561000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6482128-e20b-4533-ba72-3f7d06ebe313 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:12.561000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6482128-e20b-4533-ba72-3f7d06ebe313 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:12.562570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6482128-e20b-4533-ba72-3f7d06ebe313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:12.563065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6482128-e20b-4533-ba72-3f7d06ebe313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:12.566638 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6482128-e20b-4533-ba72-3f7d06ebe313 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:12.588546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 604/1211] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:41:12.594594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3420440-57d0-4aed-9f6a-7c3e0014b33c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:12.778473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3420440-57d0-4aed-9f6a-7c3e0014b33c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:12.781044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:12.839620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54498c9d-c40d-438c-a62b-d7597d0962d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:12.840522 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54498c9d-c40d-438c-a62b-d7597d0962d5 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:12.840838 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54498c9d-c40d-438c-a62b-d7597d0962d5 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:12.841123 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54498c9d-c40d-438c-a62b-d7597d0962d5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:12.851851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54498c9d-c40d-438c-a62b-d7597d0962d5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:12.853564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54498c9d-c40d-438c-a62b-d7597d0962d5 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-54498c9d-c40d-438c-a62b-d7597d0962d5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:41:12.870565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:12.876678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:12.877675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:12.877675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:12.877675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:41:12.877675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:41:12.877675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:41:12.877675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:41:12.877675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:41:12.877675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:41:12.877675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 17:41:12.877675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:12.877675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 608/1212] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:41:12.882342 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6482128-e20b-4533-ba72-3f7d06ebe313 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:13.050094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6482128-e20b-4533-ba72-3f7d06ebe313 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:13.103532 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc529cc-4bf6-486e-96fc-190b047b20f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:13.104967 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc529cc-4bf6-486e-96fc-190b047b20f0 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:13.105280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc529cc-4bf6-486e-96fc-190b047b20f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:13.105586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc529cc-4bf6-486e-96fc-190b047b20f0 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:13.106491 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc529cc-4bf6-486e-96fc-190b047b20f0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:13.106804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc529cc-4bf6-486e-96fc-190b047b20f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:13.108360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc529cc-4bf6-486e-96fc-190b047b20f0 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:13.138122 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 605/1213] 10.222.0.64 () {56 vars in 1143 bytes} [Thu Jul 14 17:41:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:13.145496 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54498c9d-c40d-438c-a62b-d7597d0962d5 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:13.293240 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54498c9d-c40d-438c-a62b-d7597d0962d5 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:13.339857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156630c-0b5a-4f9b-a0ed-4b93362c35a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:13.340729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156630c-0b5a-4f9b-a0ed-4b93362c35a1 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:13.341112 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156630c-0b5a-4f9b-a0ed-4b93362c35a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:13.341491 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d156630c-0b5a-4f9b-a0ed-4b93362c35a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:13.385740 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:13.437032 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156630c-0b5a-4f9b-a0ed-4b93362c35a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:13.438510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d156630c-0b5a-4f9b-a0ed-4b93362c35a1 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:13.591101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 609/1214] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:41:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:41:13.592328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc529cc-4bf6-486e-96fc-190b047b20f0 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:13.613634 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:13.712353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc529cc-4bf6-486e-96fc-190b047b20f0 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:13.762064 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae307b5-5ddf-4088-9e42-b813609c4469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:13.763048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae307b5-5ddf-4088-9e42-b813609c4469 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:13.763348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae307b5-5ddf-4088-9e42-b813609c4469 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:13.763637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae307b5-5ddf-4088-9e42-b813609c4469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:13.779278 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae307b5-5ddf-4088-9e42-b813609c4469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:13.781093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae307b5-5ddf-4088-9e42-b813609c4469 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-6ae307b5-5ddf-4088-9e42-b813609c4469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:41:13.806212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:13.807478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:13.808639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:41:13.808639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:41:13.808639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:13.808639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:13.808639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 606/1215] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:13.816803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156630c-0b5a-4f9b-a0ed-4b93362c35a1 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:13.913837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:13.978996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d156630c-0b5a-4f9b-a0ed-4b93362c35a1 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:14.031062 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82515258-3f2e-475e-be3f-1d1c29090743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:14.031986 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82515258-3f2e-475e-be3f-1d1c29090743 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:14.032402 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82515258-3f2e-475e-be3f-1d1c29090743 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:14.032820 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82515258-3f2e-475e-be3f-1d1c29090743 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:14.033681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82515258-3f2e-475e-be3f-1d1c29090743 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:14.034066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82515258-3f2e-475e-be3f-1d1c29090743 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:14.035553 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82515258-3f2e-475e-be3f-1d1c29090743 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:14.048306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 610/1216] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:14.054625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae307b5-5ddf-4088-9e42-b813609c4469 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:14.152448 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:14.205397 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae307b5-5ddf-4088-9e42-b813609c4469 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:14.254910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a11659-a360-4db9-bf37-eebfe6df21f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:14.257789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a11659-a360-4db9-bf37-eebfe6df21f0 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:14.257789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a11659-a360-4db9-bf37-eebfe6df21f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:14.257789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a11659-a360-4db9-bf37-eebfe6df21f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:14.272473 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a11659-a360-4db9-bf37-eebfe6df21f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:14.274131 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a11659-a360-4db9-bf37-eebfe6df21f0 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-a9a11659-a360-4db9-bf37-eebfe6df21f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:41:14.291377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:14.293330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:14.295850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:41:14.295850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:41:14.295850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:41:14.295850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:41:14.295850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:41:14.295850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:41:14.295850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:41:14.295850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:14.295850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 607/1217] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:41:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:41:14.299080 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82515258-3f2e-475e-be3f-1d1c29090743 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:14.470699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82515258-3f2e-475e-be3f-1d1c29090743 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:14.520097 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:14.545200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9913f60-aca7-40a9-8fc7-7a6174fc9511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:14.546644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9913f60-aca7-40a9-8fc7-7a6174fc9511 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:14.546763 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9913f60-aca7-40a9-8fc7-7a6174fc9511 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:14.547268 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9913f60-aca7-40a9-8fc7-7a6174fc9511 None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:14.549217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9913f60-aca7-40a9-8fc7-7a6174fc9511 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:14.549768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9913f60-aca7-40a9-8fc7-7a6174fc9511 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:14.552544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9913f60-aca7-40a9-8fc7-7a6174fc9511 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:14.571811 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 611/1218] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:41:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:14.578063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a11659-a360-4db9-bf37-eebfe6df21f0 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:14.748020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a11659-a360-4db9-bf37-eebfe6df21f0 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:14.807421 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:14.817881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3447b8-7719-40bc-844a-8445cb199fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:14.819565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3447b8-7719-40bc-844a-8445cb199fd6 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:14.820097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3447b8-7719-40bc-844a-8445cb199fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:14.820654 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3447b8-7719-40bc-844a-8445cb199fd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:14.822191 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3447b8-7719-40bc-844a-8445cb199fd6 None admin] RBAC: Adding query filter params (role.id=2db2bc363aa74930932c823dc291d7f3, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=619e3bec841343c5ad144f1d4e4d436d) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:14.822652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3447b8-7719-40bc-844a-8445cb199fd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:14.825826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3447b8-7719-40bc-844a-8445cb199fd6 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:14.835101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 608/1219] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:41:14 2022] GET /identity/v3/role_assignments?user.id=619e3bec841343c5ad144f1d4e4d436d&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=2db2bc363aa74930932c823dc291d7f3 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:16.942003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193be541-be69-4929-b2f1-d7b965b64a64 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:16.942618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193be541-be69-4929-b2f1-d7b965b64a64 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:16.942618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193be541-be69-4929-b2f1-d7b965b64a64 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:16.944538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 612/1220] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:41:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:41:16.950273 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67912784-6279-4845-8276-050b28af72e5 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:16.950748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67912784-6279-4845-8276-050b28af72e5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:16.950858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67912784-6279-4845-8276-050b28af72e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:16.967680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-67912784-6279-4845-8276-050b28af72e5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:41:16.968926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-67912784-6279-4845-8276-050b28af72e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:41:17.058522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 609/1221] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:41:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:41:17.098035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193be541-be69-4929-b2f1-d7b965b64a64 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:17.243977 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193be541-be69-4929-b2f1-d7b965b64a64 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:17.282259 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c3a87-1290-4ec4-b01d-195219d00624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:17.283103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c3a87-1290-4ec4-b01d-195219d00624 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:17.283400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c3a87-1290-4ec4-b01d-195219d00624 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:17.283677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c3a87-1290-4ec4-b01d-195219d00624 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:17.298367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c3a87-1290-4ec4-b01d-195219d00624 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-db0c3a87-1290-4ec4-b01d-195219d00624 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:41:17.307217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:17.308873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:17.310061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:17.310061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:17.310061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:17.310061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:17.310061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:41:17.310061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:41:17.310061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:41:17.310061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:41:17.310061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:41:17.310061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:41:17.310061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:41:17.310061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:17.310061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 613/1222] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:41:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:17.314938 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67912784-6279-4845-8276-050b28af72e5 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:17.442337 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67912784-6279-4845-8276-050b28af72e5 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:17.474186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:17.487216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaed70e-dbf7-4901-a4d5-29c299a1f1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:17.487906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaed70e-dbf7-4901-a4d5-29c299a1f1c2 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:17.488232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaed70e-dbf7-4901-a4d5-29c299a1f1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:17.488512 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaed70e-dbf7-4901-a4d5-29c299a1f1c2 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:17.489275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaed70e-dbf7-4901-a4d5-29c299a1f1c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:17.489585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaed70e-dbf7-4901-a4d5-29c299a1f1c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:17.491561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaed70e-dbf7-4901-a4d5-29c299a1f1c2 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:17.501030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 610/1223] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:41:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:17.507060 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c3a87-1290-4ec4-b01d-195219d00624 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:17.557294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:17.642953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c3a87-1290-4ec4-b01d-195219d00624 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:17.686517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0d7b0-ca94-4e0e-8fce-a9922ee226b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:17.687487 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0d7b0-ca94-4e0e-8fce-a9922ee226b2 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:17.687812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0d7b0-ca94-4e0e-8fce-a9922ee226b2 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:17.688152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0d7b0-ca94-4e0e-8fce-a9922ee226b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:17.701970 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0d7b0-ca94-4e0e-8fce-a9922ee226b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:17.703470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0d7b0-ca94-4e0e-8fce-a9922ee226b2 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-fde0d7b0-ca94-4e0e-8fce-a9922ee226b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:41:17.723300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:17.725031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:17.726241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:41:17.726241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:41:17.726241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:17.726241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:17.726241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 614/1224] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:17.732920 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaed70e-dbf7-4901-a4d5-29c299a1f1c2 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:17.874428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaed70e-dbf7-4901-a4d5-29c299a1f1c2 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:17.928771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308fd4c-230e-463a-b694-8e1c040a3f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:17.929600 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308fd4c-230e-463a-b694-8e1c040a3f38 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:17.929887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308fd4c-230e-463a-b694-8e1c040a3f38 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:17.930167 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c308fd4c-230e-463a-b694-8e1c040a3f38 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:17.930987 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308fd4c-230e-463a-b694-8e1c040a3f38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:17.931262 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308fd4c-230e-463a-b694-8e1c040a3f38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:17.933056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c308fd4c-230e-463a-b694-8e1c040a3f38 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:17.948931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 611/1225] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:17.955984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0d7b0-ca94-4e0e-8fce-a9922ee226b2 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:18.080729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:18.088973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0d7b0-ca94-4e0e-8fce-a9922ee226b2 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:18.132520 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c545834-8050-4c53-b736-7a6a2e8aa3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:18.133337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c545834-8050-4c53-b736-7a6a2e8aa3a9 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:18.133861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c545834-8050-4c53-b736-7a6a2e8aa3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:18.134186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c545834-8050-4c53-b736-7a6a2e8aa3a9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:18.143958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c545834-8050-4c53-b736-7a6a2e8aa3a9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:18.145952 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c545834-8050-4c53-b736-7a6a2e8aa3a9 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-2c545834-8050-4c53-b736-7a6a2e8aa3a9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:41:18.163219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:18.165577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:18.167724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:18.167724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:18.167724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:41:18.167724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:41:18.167724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:41:18.167724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:41:18.167724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:41:18.167724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:41:18.167724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 17:41:18.167724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:18.169657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 615/1226] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:18.170902 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308fd4c-230e-463a-b694-8e1c040a3f38 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:18.225597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:18.291688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c308fd4c-230e-463a-b694-8e1c040a3f38 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:18.338804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83673c0f-2b44-4b4d-a269-485fafc05778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:18.339541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83673c0f-2b44-4b4d-a269-485fafc05778 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:18.339831 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83673c0f-2b44-4b4d-a269-485fafc05778 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:18.340157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83673c0f-2b44-4b4d-a269-485fafc05778 None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:18.341024 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83673c0f-2b44-4b4d-a269-485fafc05778 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:18.341337 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83673c0f-2b44-4b4d-a269-485fafc05778 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:18.342937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83673c0f-2b44-4b4d-a269-485fafc05778 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:18.363457 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 612/1227] 10.222.0.64 () {56 vars in 1143 bytes} [Thu Jul 14 17:41:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:41:18.372157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c545834-8050-4c53-b736-7a6a2e8aa3a9 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:18.512743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c545834-8050-4c53-b736-7a6a2e8aa3a9 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:18.557673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0352dff-b139-4021-9343-8a73efe21359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:18.558565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0352dff-b139-4021-9343-8a73efe21359 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:18.558856 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0352dff-b139-4021-9343-8a73efe21359 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:18.559159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0352dff-b139-4021-9343-8a73efe21359 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:18.574484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0352dff-b139-4021-9343-8a73efe21359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:18.576139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0352dff-b139-4021-9343-8a73efe21359 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-d0352dff-b139-4021-9343-8a73efe21359 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:41:18.610219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:18.611409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:18.612365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:41:18.612365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:41:18.612365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:18.612365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:18.612941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 616/1228] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:18.621072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83673c0f-2b44-4b4d-a269-485fafc05778 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:18.777366 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83673c0f-2b44-4b4d-a269-485fafc05778 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:18.780661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:18.822756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd41fa-d000-4e32-a0ef-245bad4aa2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:18.823440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd41fa-d000-4e32-a0ef-245bad4aa2a1 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:18.823736 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd41fa-d000-4e32-a0ef-245bad4aa2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:18.824033 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbd41fa-d000-4e32-a0ef-245bad4aa2a1 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:18.824770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd41fa-d000-4e32-a0ef-245bad4aa2a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:18.825147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd41fa-d000-4e32-a0ef-245bad4aa2a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:18.826446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbd41fa-d000-4e32-a0ef-245bad4aa2a1 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:18.839964 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 613/1229] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:18.846358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0352dff-b139-4021-9343-8a73efe21359 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:18.856884 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:18.984668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0352dff-b139-4021-9343-8a73efe21359 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:19.044277 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345ea6c-6392-4f2f-9fb2-2a1d28c9c691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:19.045196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345ea6c-6392-4f2f-9fb2-2a1d28c9c691 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:19.045659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345ea6c-6392-4f2f-9fb2-2a1d28c9c691 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:19.045950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345ea6c-6392-4f2f-9fb2-2a1d28c9c691 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:19.054124 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345ea6c-6392-4f2f-9fb2-2a1d28c9c691 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:19.056316 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b345ea6c-6392-4f2f-9fb2-2a1d28c9c691 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-b345ea6c-6392-4f2f-9fb2-2a1d28c9c691 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:41:19.076358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:19.077852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:19.079121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:41:19.079121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:41:19.079121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:41:19.079121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:41:19.079121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:41:19.079121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:41:19.079121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:41:19.079121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:19.079121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 617/1230] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:41:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:41:19.085514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd41fa-d000-4e32-a0ef-245bad4aa2a1 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:19.218103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbd41fa-d000-4e32-a0ef-245bad4aa2a1 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:19.262247 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a5bf8-eb9c-4fe5-a8b2-1ad28a2f0067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:19.263486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a5bf8-eb9c-4fe5-a8b2-1ad28a2f0067 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:19.263486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a5bf8-eb9c-4fe5-a8b2-1ad28a2f0067 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:19.263486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a5bf8-eb9c-4fe5-a8b2-1ad28a2f0067 None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:19.264433 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a5bf8-eb9c-4fe5-a8b2-1ad28a2f0067 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:19.264683 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a5bf8-eb9c-4fe5-a8b2-1ad28a2f0067 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:19.266120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a5bf8-eb9c-4fe5-a8b2-1ad28a2f0067 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:19.280414 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 614/1231] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:41:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:19.287875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345ea6c-6392-4f2f-9fb2-2a1d28c9c691 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:19.375894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:19.422674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345ea6c-6392-4f2f-9fb2-2a1d28c9c691 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:19.476615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5b7df-d29d-4427-9879-48fe0bc54f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:19.477484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5b7df-d29d-4427-9879-48fe0bc54f90 None admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:19.477805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5b7df-d29d-4427-9879-48fe0bc54f90 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:19.478067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5b7df-d29d-4427-9879-48fe0bc54f90 None admin] PATH_INFO: `/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/619e3bec841343c5ad144f1d4e4d436d/roles/2db2bc363aa74930932c823dc291d7f3` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:19.494781 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5b7df-d29d-4427-9879-48fe0bc54f90 None admin] RBAC: Authorizing `identity:create_grant(project_id=de62d2fb17f846f2bce5bef108353fd8, user_id=619e3bec841343c5ad144f1d4e4d436d, role_id=2db2bc363aa74930932c823dc291d7f3)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:19.496367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5b7df-d29d-4427-9879-48fe0bc54f90 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:19.511727 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 618/1232] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:41:19 2022] PUT /identity/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/619e3bec841343c5ad144f1d4e4d436d/roles/2db2bc363aa74930932c823dc291d7f3 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:41:21.840098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa2eaa-a3ba-41c1-848b-bea820345f7f None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:21.840974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa2eaa-a3ba-41c1-848b-bea820345f7f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:21.841138 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa2eaa-a3ba-41c1-848b-bea820345f7f None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:21.843306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 615/1233] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:41:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:41:21.854923 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735726b-4b47-4e54-86a0-c043977bb852 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:21.855690 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735726b-4b47-4e54-86a0-c043977bb852 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:21.855901 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735726b-4b47-4e54-86a0-c043977bb852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:21.884892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4735726b-4b47-4e54-86a0-c043977bb852 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:41:21.887508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4735726b-4b47-4e54-86a0-c043977bb852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:41:22.023806 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 619/1234] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:41:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:41:22.059798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa2eaa-a3ba-41c1-848b-bea820345f7f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:22.103442 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:22.197826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa2eaa-a3ba-41c1-848b-bea820345f7f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:22.251639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a1a3b-5606-473c-9fc2-e289ac1c9783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:22.252486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a1a3b-5606-473c-9fc2-e289ac1c9783 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:22.252772 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a1a3b-5606-473c-9fc2-e289ac1c9783 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:22.253039 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a1a3b-5606-473c-9fc2-e289ac1c9783 None admin] PATH_INFO: `/v3/roles/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:22.265882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a1a3b-5606-473c-9fc2-e289ac1c9783 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-ee9a1a3b-5606-473c-9fc2-e289ac1c9783 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:41:22.274440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:22.276504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:22.278076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:22.278076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:22.278076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:22.278076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:22.278076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:41:22.278076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:41:22.278076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:41:22.278076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:41:22.278076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:41:22.278076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:41:22.278076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 17:41:22.278076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:22.278076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 616/1235] 10.222.0.64 () {56 vars in 1102 bytes} [Thu Jul 14 17:41:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:22.282243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735726b-4b47-4e54-86a0-c043977bb852 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:22.341001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:22.431052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735726b-4b47-4e54-86a0-c043977bb852 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:22.486587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121ffa0-2bb6-414a-8320-677bc50dc950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:22.487540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121ffa0-2bb6-414a-8320-677bc50dc950 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:22.488329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121ffa0-2bb6-414a-8320-677bc50dc950 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:22.488839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121ffa0-2bb6-414a-8320-677bc50dc950 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:22.490128 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121ffa0-2bb6-414a-8320-677bc50dc950 None admin] RBAC: Adding query filter params (name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:22.491401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121ffa0-2bb6-414a-8320-677bc50dc950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:22.494810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121ffa0-2bb6-414a-8320-677bc50dc950 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:22.508255 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 620/1236] 10.222.0.64 () {56 vars in 1103 bytes} [Thu Jul 14 17:41:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:41:22.515293 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a1a3b-5606-473c-9fc2-e289ac1c9783 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:22.652129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:22.679573 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a1a3b-5606-473c-9fc2-e289ac1c9783 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:22.728395 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6227c-c342-4ddd-8dce-1a583a70b08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:22.729385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6227c-c342-4ddd-8dce-1a583a70b08c None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:22.729769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6227c-c342-4ddd-8dce-1a583a70b08c None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:22.730169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6227c-c342-4ddd-8dce-1a583a70b08c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:22.747470 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6227c-c342-4ddd-8dce-1a583a70b08c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:22.749513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6227c-c342-4ddd-8dce-1a583a70b08c None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-73d6227c-c342-4ddd-8dce-1a583a70b08c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:41:22.779657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:22.781756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:22.790631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:41:22.790631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:41:22.790631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:22.790631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:22.790631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 617/1237] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:22.791348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121ffa0-2bb6-414a-8320-677bc50dc950 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:22.929326 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121ffa0-2bb6-414a-8320-677bc50dc950 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:22.942767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:22.985320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd20ce-aa67-41ac-a12a-f023a327243b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:22.986042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd20ce-aa67-41ac-a12a-f023a327243b None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:22.986329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd20ce-aa67-41ac-a12a-f023a327243b None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:22.986679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dd20ce-aa67-41ac-a12a-f023a327243b None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:22.987523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd20ce-aa67-41ac-a12a-f023a327243b None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:22.987855 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd20ce-aa67-41ac-a12a-f023a327243b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:22.989999 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dd20ce-aa67-41ac-a12a-f023a327243b None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:23.008351 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 621/1238] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:41:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:23.015649 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6227c-c342-4ddd-8dce-1a583a70b08c None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:23.163185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6227c-c342-4ddd-8dce-1a583a70b08c None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:23.207073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb6721-8a94-4ec5-b132-88b80140e906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:23.207854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb6721-8a94-4ec5-b132-88b80140e906 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:23.208179 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb6721-8a94-4ec5-b132-88b80140e906 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:23.208461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb6721-8a94-4ec5-b132-88b80140e906 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:23.212351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:23.220597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb6721-8a94-4ec5-b132-88b80140e906 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:23.222758 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb6721-8a94-4ec5-b132-88b80140e906 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-91cb6721-8a94-4ec5-b132-88b80140e906 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:41:23.241525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:23.242993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:23.244780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:23.244780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:23.244780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:41:23.244780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:41:23.244780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:41:23.244780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:41:23.244780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:41:23.244780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:41:23.244780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 17:41:23.244780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:23.244780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 618/1239] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:41:23.249008 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd20ce-aa67-41ac-a12a-f023a327243b None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:23.390019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dd20ce-aa67-41ac-a12a-f023a327243b None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:23.434686 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425f0ef-082e-410c-bd4e-bbf60b725b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:23.435436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425f0ef-082e-410c-bd4e-bbf60b725b0b None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:23.435717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425f0ef-082e-410c-bd4e-bbf60b725b0b None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:23.436101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0425f0ef-082e-410c-bd4e-bbf60b725b0b None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:23.436917 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425f0ef-082e-410c-bd4e-bbf60b725b0b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:23.437228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425f0ef-082e-410c-bd4e-bbf60b725b0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:23.438681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0425f0ef-082e-410c-bd4e-bbf60b725b0b None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:23.460397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 622/1240] 10.222.0.64 () {56 vars in 1143 bytes} [Thu Jul 14 17:41:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:23.468250 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb6721-8a94-4ec5-b132-88b80140e906 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:23.601870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb6721-8a94-4ec5-b132-88b80140e906 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:23.653108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2734ba9-0ff2-419c-9ae9-fc12ed0cab7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:23.653777 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2734ba9-0ff2-419c-9ae9-fc12ed0cab7e None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:23.654054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2734ba9-0ff2-419c-9ae9-fc12ed0cab7e None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:23.654329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2734ba9-0ff2-419c-9ae9-fc12ed0cab7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:23.730687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:23.739451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2734ba9-0ff2-419c-9ae9-fc12ed0cab7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:23.741792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2734ba9-0ff2-419c-9ae9-fc12ed0cab7e None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:23.896802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 619/1241] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:41:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:41:23.902808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425f0ef-082e-410c-bd4e-bbf60b725b0b None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:23.938117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:24.045025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0425f0ef-082e-410c-bd4e-bbf60b725b0b None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:24.087349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9db45-2a00-4a43-a29b-fceef48bff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:24.088202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9db45-2a00-4a43-a29b-fceef48bff49 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:24.088486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9db45-2a00-4a43-a29b-fceef48bff49 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:24.088757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9db45-2a00-4a43-a29b-fceef48bff49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:24.101356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9db45-2a00-4a43-a29b-fceef48bff49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:24.103032 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9db45-2a00-4a43-a29b-fceef48bff49 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-c7b9db45-2a00-4a43-a29b-fceef48bff49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:41:24.130086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:24.131279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:24.132382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:41:24.132382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:41:24.132382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:24.132382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:24.132382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 623/1242] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:41:24.139262 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2734ba9-0ff2-419c-9ae9-fc12ed0cab7e None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:24.255418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:24.262016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2734ba9-0ff2-419c-9ae9-fc12ed0cab7e None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:24.303764 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d872d3d-f2e3-438a-80e6-5d3054e9abdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:24.305720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d872d3d-f2e3-438a-80e6-5d3054e9abdf None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:24.306352 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d872d3d-f2e3-438a-80e6-5d3054e9abdf None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:24.307025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d872d3d-f2e3-438a-80e6-5d3054e9abdf None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:24.309557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d872d3d-f2e3-438a-80e6-5d3054e9abdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:24.310151 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d872d3d-f2e3-438a-80e6-5d3054e9abdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:24.311762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d872d3d-f2e3-438a-80e6-5d3054e9abdf None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:24.325184 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 620/1243] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:41:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:24.333594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9db45-2a00-4a43-a29b-fceef48bff49 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:24.439869 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:24.485087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9db45-2a00-4a43-a29b-fceef48bff49 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:24.530729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356eb408-26ca-4269-a6f0-a0105f74608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:24.531469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356eb408-26ca-4269-a6f0-a0105f74608a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:24.531754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356eb408-26ca-4269-a6f0-a0105f74608a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:24.532063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356eb408-26ca-4269-a6f0-a0105f74608a None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:24.540048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356eb408-26ca-4269-a6f0-a0105f74608a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:24.541470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356eb408-26ca-4269-a6f0-a0105f74608a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-356eb408-26ca-4269-a6f0-a0105f74608a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:41:24.559495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:24.560729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:24.561905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:41:24.561905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:41:24.561905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:41:24.561905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:41:24.561905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:41:24.561905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:41:24.561905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:41:24.561905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:24.561905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 624/1244] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:41:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:24.566878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d872d3d-f2e3-438a-80e6-5d3054e9abdf None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:24.687360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d872d3d-f2e3-438a-80e6-5d3054e9abdf None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:24.732252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d8bc1c-911a-44b4-9f52-62dd8e3d6dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:24.732933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8bc1c-911a-44b4-9f52-62dd8e3d6dcc None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:24.733218 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8bc1c-911a-44b4-9f52-62dd8e3d6dcc None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:24.733498 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d8bc1c-911a-44b4-9f52-62dd8e3d6dcc None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:24.734294 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8bc1c-911a-44b4-9f52-62dd8e3d6dcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:24.734604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8bc1c-911a-44b4-9f52-62dd8e3d6dcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:24.736097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d8bc1c-911a-44b4-9f52-62dd8e3d6dcc None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:24.750632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 621/1245] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:41:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:41:24.757362 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356eb408-26ca-4269-a6f0-a0105f74608a None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:24.926813 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356eb408-26ca-4269-a6f0-a0105f74608a None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:24.968319 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd98ed14-b0c2-46ee-8de7-d46ec9017556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:24.969414 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98ed14-b0c2-46ee-8de7-d46ec9017556 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:24.969414 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98ed14-b0c2-46ee-8de7-d46ec9017556 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:24.969414 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd98ed14-b0c2-46ee-8de7-d46ec9017556 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:24.970443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98ed14-b0c2-46ee-8de7-d46ec9017556 None admin] RBAC: Adding query filter params (role.id=2db2bc363aa74930932c823dc291d7f3, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=619e3bec841343c5ad144f1d4e4d436d) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:24.970862 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98ed14-b0c2-46ee-8de7-d46ec9017556 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:24.972327 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd98ed14-b0c2-46ee-8de7-d46ec9017556 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:24.979729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 625/1246] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:41:24 2022] GET /identity/v3/role_assignments?user.id=619e3bec841343c5ad144f1d4e4d436d&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=2db2bc363aa74930932c823dc291d7f3 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:41:27.163403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844d07a-f585-487d-849a-f17c1468606c None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:27.164016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844d07a-f585-487d-849a-f17c1468606c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:27.164127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2844d07a-f585-487d-849a-f17c1468606c None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:27.165652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 622/1247] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:41:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:41:27.172382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be10d8b-ef16-4cc9-8200-7ffd1536eb09 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:27.173297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be10d8b-ef16-4cc9-8200-7ffd1536eb09 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:27.173297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be10d8b-ef16-4cc9-8200-7ffd1536eb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:27.180314 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:27.199038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-6be10d8b-ef16-4cc9-8200-7ffd1536eb09 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:41:27.201153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6be10d8b-ef16-4cc9-8200-7ffd1536eb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:41:27.299522 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 626/1248] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:41:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:41:27.329628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844d07a-f585-487d-849a-f17c1468606c None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:27.354310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:27.461192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2844d07a-f585-487d-849a-f17c1468606c None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:27.502660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4420b-0f31-4d9b-a37a-23432ce2a139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:27.503506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4420b-0f31-4d9b-a37a-23432ce2a139 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:27.503789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4420b-0f31-4d9b-a37a-23432ce2a139 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:27.504087 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4420b-0f31-4d9b-a37a-23432ce2a139 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:27.517472 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4420b-0f31-4d9b-a37a-23432ce2a139 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-ebb4420b-0f31-4d9b-a37a-23432ce2a139 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:41:27.526307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:27.527556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:27.528612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:27.528612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:27.528612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:27.528612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:27.528612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:41:27.528612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:41:27.528612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:41:27.528612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:41:27.528612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:41:27.528612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:41:27.528612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:41:27.528612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:27.528612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 623/1249] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:41:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:27.534071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be10d8b-ef16-4cc9-8200-7ffd1536eb09 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:27.664257 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be10d8b-ef16-4cc9-8200-7ffd1536eb09 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:27.703296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:27.712475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d2b8e-9f90-452b-b8ff-c4a52f769955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:27.713602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d2b8e-9f90-452b-b8ff-c4a52f769955 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:27.714034 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d2b8e-9f90-452b-b8ff-c4a52f769955 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:27.714500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d2b8e-9f90-452b-b8ff-c4a52f769955 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:27.715597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d2b8e-9f90-452b-b8ff-c4a52f769955 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:27.716025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d2b8e-9f90-452b-b8ff-c4a52f769955 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:27.717934 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d2b8e-9f90-452b-b8ff-c4a52f769955 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:27.730081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 627/1250] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:41:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:27.736351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4420b-0f31-4d9b-a37a-23432ce2a139 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:27.845682 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:27.882964 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4420b-0f31-4d9b-a37a-23432ce2a139 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:27.929987 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cfae1-5721-49cf-8006-5d4c152156f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:27.930798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cfae1-5721-49cf-8006-5d4c152156f1 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:27.931080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cfae1-5721-49cf-8006-5d4c152156f1 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:27.931361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1cfae1-5721-49cf-8006-5d4c152156f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:27.944030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cfae1-5721-49cf-8006-5d4c152156f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:27.946257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1cfae1-5721-49cf-8006-5d4c152156f1 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-1b1cfae1-5721-49cf-8006-5d4c152156f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:41:27.967265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:27.968449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:27.969363 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:41:27.969363 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:41:27.969363 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:27.969363 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:27.969363 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 624/1251] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:27.974633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d2b8e-9f90-452b-b8ff-c4a52f769955 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:28.100866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d2b8e-9f90-452b-b8ff-c4a52f769955 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:28.141975 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd30107-d56d-4842-8afa-47d848b8fdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:28.143009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd30107-d56d-4842-8afa-47d848b8fdd3 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:28.143534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd30107-d56d-4842-8afa-47d848b8fdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:28.143951 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd30107-d56d-4842-8afa-47d848b8fdd3 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:28.144812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd30107-d56d-4842-8afa-47d848b8fdd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:28.145145 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd30107-d56d-4842-8afa-47d848b8fdd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:28.146672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd30107-d56d-4842-8afa-47d848b8fdd3 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:28.161652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 628/1252] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:28.167747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cfae1-5721-49cf-8006-5d4c152156f1 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:28.300036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1cfae1-5721-49cf-8006-5d4c152156f1 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:28.331608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:28.341733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450aabbf-0fd4-49ae-8c1b-420d41d5b097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:28.342581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450aabbf-0fd4-49ae-8c1b-420d41d5b097 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:28.342869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450aabbf-0fd4-49ae-8c1b-420d41d5b097 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:28.343214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450aabbf-0fd4-49ae-8c1b-420d41d5b097 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:28.352715 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450aabbf-0fd4-49ae-8c1b-420d41d5b097 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:28.354632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450aabbf-0fd4-49ae-8c1b-420d41d5b097 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-450aabbf-0fd4-49ae-8c1b-420d41d5b097 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:41:28.370393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:28.371668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:28.372931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:28.372931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:28.372931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:41:28.372931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:41:28.372931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:41:28.372931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:41:28.372931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:41:28.372931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:41:28.372931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 17:41:28.372931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:28.378158 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 625/1253] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:28.378301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd30107-d56d-4842-8afa-47d848b8fdd3 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:28.448561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:28.500271 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd30107-d56d-4842-8afa-47d848b8fdd3 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:28.555678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd409cf-ef36-47df-85a8-be88686a9581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:28.556223 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd409cf-ef36-47df-85a8-be88686a9581 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:28.556514 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd409cf-ef36-47df-85a8-be88686a9581 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:28.556783 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd409cf-ef36-47df-85a8-be88686a9581 None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:28.557654 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd409cf-ef36-47df-85a8-be88686a9581 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:28.557947 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd409cf-ef36-47df-85a8-be88686a9581 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:28.559396 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd409cf-ef36-47df-85a8-be88686a9581 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:28.578230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 629/1254] 10.222.0.64 () {56 vars in 1143 bytes} [Thu Jul 14 17:41:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:28.585094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450aabbf-0fd4-49ae-8c1b-420d41d5b097 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:28.710121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450aabbf-0fd4-49ae-8c1b-420d41d5b097 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:28.757763 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8ff57-e0e0-48a2-8c51-796696be3551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:28.758594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8ff57-e0e0-48a2-8c51-796696be3551 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:28.759007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8ff57-e0e0-48a2-8c51-796696be3551 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:28.759577 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8ff57-e0e0-48a2-8c51-796696be3551 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:28.864722 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:28.882555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8ff57-e0e0-48a2-8c51-796696be3551 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:28.884139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8ff57-e0e0-48a2-8c51-796696be3551 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:29.019826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 626/1255] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:41:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:41:29.026531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd409cf-ef36-47df-85a8-be88686a9581 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:29.161902 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd409cf-ef36-47df-85a8-be88686a9581 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:29.177535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:29.204199 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c367b9b-d18e-4262-9579-b0192085c763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:29.205041 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c367b9b-d18e-4262-9579-b0192085c763 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:29.205327 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c367b9b-d18e-4262-9579-b0192085c763 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:29.205766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c367b9b-d18e-4262-9579-b0192085c763 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:29.218991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c367b9b-d18e-4262-9579-b0192085c763 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:29.220730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c367b9b-d18e-4262-9579-b0192085c763 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-2c367b9b-d18e-4262-9579-b0192085c763 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:41:29.243973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:29.245461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:29.246603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:41:29.246603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:41:29.246603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:29.246603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:29.251991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 630/1256] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 17:41:29.252318 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8ff57-e0e0-48a2-8c51-796696be3551 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:29.363418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:29.386119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8ff57-e0e0-48a2-8c51-796696be3551 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:29.425580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a77c8-8e3f-418c-a00f-b7cb42a00348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:29.426261 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a77c8-8e3f-418c-a00f-b7cb42a00348 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:29.426552 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a77c8-8e3f-418c-a00f-b7cb42a00348 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:29.426863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a77c8-8e3f-418c-a00f-b7cb42a00348 None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:29.427599 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a77c8-8e3f-418c-a00f-b7cb42a00348 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:29.427908 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a77c8-8e3f-418c-a00f-b7cb42a00348 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:29.429415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a77c8-8e3f-418c-a00f-b7cb42a00348 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:29.451389 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 627/1257] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:41:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:29.452710 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c367b9b-d18e-4262-9579-b0192085c763 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:29.594197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c367b9b-d18e-4262-9579-b0192085c763 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:29.635264 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e0573-a2dd-4d25-976d-22bed7bef7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:29.636365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e0573-a2dd-4d25-976d-22bed7bef7f7 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:29.636584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e0573-a2dd-4d25-976d-22bed7bef7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:29.636863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e0573-a2dd-4d25-976d-22bed7bef7f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:29.644592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e0573-a2dd-4d25-976d-22bed7bef7f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:29.646097 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183e0573-a2dd-4d25-976d-22bed7bef7f7 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-183e0573-a2dd-4d25-976d-22bed7bef7f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:41:29.664191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:29.665336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:29.666230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:41:29.666230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:41:29.666230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:41:29.666230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:41:29.666230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:41:29.666230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:41:29.666230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:41:29.666230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:29.666230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 631/1258] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:41:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:29.672434 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a77c8-8e3f-418c-a00f-b7cb42a00348 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:29.817034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a77c8-8e3f-418c-a00f-b7cb42a00348 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:29.869761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eed079-52b5-439a-be15-22af9c1b649f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:29.870907 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed079-52b5-439a-be15-22af9c1b649f None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:29.870907 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed079-52b5-439a-be15-22af9c1b649f None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:29.871170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eed079-52b5-439a-be15-22af9c1b649f None admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:29.871914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eed079-52b5-439a-be15-22af9c1b649f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:29.872140 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eed079-52b5-439a-be15-22af9c1b649f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:29.873591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eed079-52b5-439a-be15-22af9c1b649f None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:29.889220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 628/1259] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:41:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:29.895544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e0573-a2dd-4d25-976d-22bed7bef7f7 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:29.930189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:30.025341 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e0573-a2dd-4d25-976d-22bed7bef7f7 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:30.076196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3808e7-f2b9-483a-8738-e6b5a7ca2c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:30.076885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3808e7-f2b9-483a-8738-e6b5a7ca2c7c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:30.077160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3808e7-f2b9-483a-8738-e6b5a7ca2c7c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:30.077436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3808e7-f2b9-483a-8738-e6b5a7ca2c7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:30.078290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3808e7-f2b9-483a-8738-e6b5a7ca2c7c None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=619e3bec841343c5ad144f1d4e4d436d) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:30.078692 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3808e7-f2b9-483a-8738-e6b5a7ca2c7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:30.080178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3808e7-f2b9-483a-8738-e6b5a7ca2c7c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:30.086669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 632/1260] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:41:29 2022] GET /identity/v3/role_assignments?user.id=619e3bec841343c5ad144f1d4e4d436d&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:32.127993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc8ec9-abd2-4904-bf9a-f39df24c1edd None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:32.128488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc8ec9-abd2-4904-bf9a-f39df24c1edd None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:32.128591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc8ec9-abd2-4904-bf9a-f39df24c1edd None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:32.129947 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 629/1261] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:41:32.135899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4861-ff46-4b8d-95a1-e895f9246be8 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:32.136360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4861-ff46-4b8d-95a1-e895f9246be8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:32.136477 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee4861-ff46-4b8d-95a1-e895f9246be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:32.154356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5dee4861-ff46-4b8d-95a1-e895f9246be8 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:41:32.155559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5dee4861-ff46-4b8d-95a1-e895f9246be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:41:32.223622 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 633/1262] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:41:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:41:32.248892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc8ec9-abd2-4904-bf9a-f39df24c1edd None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:32.261782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:32.357818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc8ec9-abd2-4904-bf9a-f39df24c1edd None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:32.394332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1283cb9-328c-4ee3-9b35-9eb7fca26aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:32.395048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1283cb9-328c-4ee3-9b35-9eb7fca26aad None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:32.395334 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1283cb9-328c-4ee3-9b35-9eb7fca26aad None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:32.395614 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1283cb9-328c-4ee3-9b35-9eb7fca26aad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:32.405878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1283cb9-328c-4ee3-9b35-9eb7fca26aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-d1283cb9-328c-4ee3-9b35-9eb7fca26aad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:41:32.413556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:32.414703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:32.415796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:32.415796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:32.415796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:32.415796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:32.415796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:41:32.415796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:41:32.415796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:41:32.415796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:41:32.415796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:41:32.415796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:41:32.415796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:41:32.415796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:32.415796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 630/1263] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:41:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:32.421090 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee4861-ff46-4b8d-95a1-e895f9246be8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:32.441313 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:32.523134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee4861-ff46-4b8d-95a1-e895f9246be8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:32.557843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a538b0-fd31-47a0-b316-41aad1789328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:32.558509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a538b0-fd31-47a0-b316-41aad1789328 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:32.558793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a538b0-fd31-47a0-b316-41aad1789328 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:32.559072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a538b0-fd31-47a0-b316-41aad1789328 None admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:32.559833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a538b0-fd31-47a0-b316-41aad1789328 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:32.560164 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a538b0-fd31-47a0-b316-41aad1789328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:32.561533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a538b0-fd31-47a0-b316-41aad1789328 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:32.569747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 634/1264] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:41:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:32.575474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1283cb9-328c-4ee3-9b35-9eb7fca26aad None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:32.641335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:32.675279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1283cb9-328c-4ee3-9b35-9eb7fca26aad None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:32.708010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dc984-a283-4aae-bfbe-386f373b8251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:32.708807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dc984-a283-4aae-bfbe-386f373b8251 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:32.709093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dc984-a283-4aae-bfbe-386f373b8251 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:32.709371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3dc984-a283-4aae-bfbe-386f373b8251 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:32.719818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dc984-a283-4aae-bfbe-386f373b8251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:32.721294 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3dc984-a283-4aae-bfbe-386f373b8251 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-4b3dc984-a283-4aae-bfbe-386f373b8251 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:41:32.738547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:32.739687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:32.740699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:41:32.740699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:41:32.740699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:32.740699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:32.740699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 631/1265] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:32.746354 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a538b0-fd31-47a0-b316-41aad1789328 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:32.835069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:32.853919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a538b0-fd31-47a0-b316-41aad1789328 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:32.886334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10e97d-df02-4027-94e9-bedb283a7ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:32.886985 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10e97d-df02-4027-94e9-bedb283a7ec6 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:32.887268 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10e97d-df02-4027-94e9-bedb283a7ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:32.887541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10e97d-df02-4027-94e9-bedb283a7ec6 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:32.888288 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10e97d-df02-4027-94e9-bedb283a7ec6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:32.888597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10e97d-df02-4027-94e9-bedb283a7ec6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:32.889956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10e97d-df02-4027-94e9-bedb283a7ec6 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:32.901705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 635/1266] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:41:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:32.907545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dc984-a283-4aae-bfbe-386f373b8251 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:33.007828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3dc984-a283-4aae-bfbe-386f373b8251 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:33.027541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:33.041775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f80df1-6aec-4840-9c6a-8caf8f5381b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:33.042588 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80df1-6aec-4840-9c6a-8caf8f5381b7 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:33.042876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80df1-6aec-4840-9c6a-8caf8f5381b7 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:33.043157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f80df1-6aec-4840-9c6a-8caf8f5381b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:33.051396 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f80df1-6aec-4840-9c6a-8caf8f5381b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:33.052909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f80df1-6aec-4840-9c6a-8caf8f5381b7 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-64f80df1-6aec-4840-9c6a-8caf8f5381b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:41:33.067616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:33.068779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:33.070529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:33.070529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:33.070529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:41:33.070529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:41:33.070529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:41:33.070529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:41:33.070529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:41:33.070529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:41:33.070529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 17:41:33.070529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:33.070529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 632/1267] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:33.075320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10e97d-df02-4027-94e9-bedb283a7ec6 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:33.173519 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10e97d-df02-4027-94e9-bedb283a7ec6 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:33.206679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c80a9-d908-4e47-97b6-e905a1f9917a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:33.207348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c80a9-d908-4e47-97b6-e905a1f9917a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:33.207634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c80a9-d908-4e47-97b6-e905a1f9917a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:33.207913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c80a9-d908-4e47-97b6-e905a1f9917a None admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:33.208775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c80a9-d908-4e47-97b6-e905a1f9917a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:33.209088 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c80a9-d908-4e47-97b6-e905a1f9917a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:33.210509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c80a9-d908-4e47-97b6-e905a1f9917a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:33.218833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:33.227354 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 636/1268] 10.222.0.64 () {56 vars in 1143 bytes} [Thu Jul 14 17:41:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:33.233088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f80df1-6aec-4840-9c6a-8caf8f5381b7 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:33.331294 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f80df1-6aec-4840-9c6a-8caf8f5381b7 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:33.364565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba6957-cea5-4825-9e9c-68315ef6f995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:33.365361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba6957-cea5-4825-9e9c-68315ef6f995 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:33.365643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba6957-cea5-4825-9e9c-68315ef6f995 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:33.366545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecba6957-cea5-4825-9e9c-68315ef6f995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:33.377029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecba6957-cea5-4825-9e9c-68315ef6f995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:33.378481 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecba6957-cea5-4825-9e9c-68315ef6f995 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-ecba6957-cea5-4825-9e9c-68315ef6f995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:41:33.395637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:33.396828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:33.397748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:41:33.397748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:41:33.397748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:33.397748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:33.397748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 633/1269] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:33.403128 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c80a9-d908-4e47-97b6-e905a1f9917a None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:33.501853 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c80a9-d908-4e47-97b6-e905a1f9917a None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:33.534214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f46ae9-97bc-4a6c-ab47-84613abc06f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:33.534879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f46ae9-97bc-4a6c-ab47-84613abc06f0 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:33.535164 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f46ae9-97bc-4a6c-ab47-84613abc06f0 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:33.535444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f46ae9-97bc-4a6c-ab47-84613abc06f0 None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:33.536198 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f46ae9-97bc-4a6c-ab47-84613abc06f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:33.536511 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f46ae9-97bc-4a6c-ab47-84613abc06f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:33.538428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f46ae9-97bc-4a6c-ab47-84613abc06f0 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:33.549338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 637/1270] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:41:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:33.555700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba6957-cea5-4825-9e9c-68315ef6f995 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:33.586189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:33.653789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecba6957-cea5-4825-9e9c-68315ef6f995 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:33.686441 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b9954-9cbe-466c-a0f7-403bb3112507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:33.687225 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b9954-9cbe-466c-a0f7-403bb3112507 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:33.687508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b9954-9cbe-466c-a0f7-403bb3112507 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:33.687783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b9954-9cbe-466c-a0f7-403bb3112507 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:33.694285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b9954-9cbe-466c-a0f7-403bb3112507 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:33.695706 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b9954-9cbe-466c-a0f7-403bb3112507 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-9f9b9954-9cbe-466c-a0f7-403bb3112507 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:41:33.710136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:33.711147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:33.712446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:41:33.712446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:41:33.712446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:41:33.712446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:41:33.712446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:41:33.712446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:41:33.712446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:41:33.712446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:33.712446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 634/1271] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:41:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:33.717105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f46ae9-97bc-4a6c-ab47-84613abc06f0 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:33.779719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:33.820994 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f46ae9-97bc-4a6c-ab47-84613abc06f0 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:33.854275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff198d-1c25-4f04-a98e-2f038348263c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:33.854854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff198d-1c25-4f04-a98e-2f038348263c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:33.855137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff198d-1c25-4f04-a98e-2f038348263c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:33.855414 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff198d-1c25-4f04-a98e-2f038348263c None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:33.856232 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff198d-1c25-4f04-a98e-2f038348263c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:33.856541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff198d-1c25-4f04-a98e-2f038348263c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:33.857958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff198d-1c25-4f04-a98e-2f038348263c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:33.869746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 638/1272] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:41:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:33.875851 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b9954-9cbe-466c-a0f7-403bb3112507 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:33.966133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:33.974820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b9954-9cbe-466c-a0f7-403bb3112507 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:34.008267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f8a3fd-48a6-490b-8dde-4f931cbe580a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:34.009152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a3fd-48a6-490b-8dde-4f931cbe580a None admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:34.009438 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a3fd-48a6-490b-8dde-4f931cbe580a None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:34.009972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f8a3fd-48a6-490b-8dde-4f931cbe580a None admin] PATH_INFO: `/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/619e3bec841343c5ad144f1d4e4d436d/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:34.011977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8a3fd-48a6-490b-8dde-4f931cbe580a None admin] RBAC: Authorizing `identity:create_grant(project_id=de62d2fb17f846f2bce5bef108353fd8, user_id=619e3bec841343c5ad144f1d4e4d436d, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:34.013389 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f8a3fd-48a6-490b-8dde-4f931cbe580a None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:34.026501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 635/1273] 10.222.0.64 () {58 vars in 1439 bytes} [Thu Jul 14 17:41:33 2022] PUT /identity/v3/projects/de62d2fb17f846f2bce5bef108353fd8/users/619e3bec841343c5ad144f1d4e4d436d/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:41:36.071050 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedea10c-382f-4fd5-a257-60e116886c50 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:36.071618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedea10c-382f-4fd5-a257-60e116886c50 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:36.071618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedea10c-382f-4fd5-a257-60e116886c50 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:36.073012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 639/1274] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:41:36.078105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23c60a-f308-4f5e-8913-5dd544cb6a09 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:36.078491 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23c60a-f308-4f5e-8913-5dd544cb6a09 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:36.078669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23c60a-f308-4f5e-8913-5dd544cb6a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:36.095673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-9e23c60a-f308-4f5e-8913-5dd544cb6a09 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:41:36.096879 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9e23c60a-f308-4f5e-8913-5dd544cb6a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:41:36.161694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 636/1275] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:41:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:41:36.185691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedea10c-382f-4fd5-a257-60e116886c50 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:36.283464 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedea10c-382f-4fd5-a257-60e116886c50 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:36.300528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:36.317110 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf58f08-2f08-48fa-9760-25d8881198ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:36.317899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf58f08-2f08-48fa-9760-25d8881198ce None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:36.318184 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf58f08-2f08-48fa-9760-25d8881198ce None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:36.318463 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf58f08-2f08-48fa-9760-25d8881198ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:36.328967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf58f08-2f08-48fa-9760-25d8881198ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-ecf58f08-2f08-48fa-9760-25d8881198ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 17:41:36.337234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:36.338353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:36.339233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:36.339233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:36.339233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:36.339233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:36.339233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 17:41:36.339233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 17:41:36.339233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 17:41:36.339233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 17:41:36.339233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 17:41:36.339233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 17:41:36.339233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 17:41:36.339233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:36.339233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 640/1276] 10.222.0.64 () {56 vars in 1096 bytes} [Thu Jul 14 17:41:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:36.344631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23c60a-f308-4f5e-8913-5dd544cb6a09 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:36.424579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:36.443277 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23c60a-f308-4f5e-8913-5dd544cb6a09 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:36.484876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02df61-0b18-489b-9e68-1f38964d2f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:36.485555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02df61-0b18-489b-9e68-1f38964d2f63 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:36.485842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02df61-0b18-489b-9e68-1f38964d2f63 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:36.486121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02df61-0b18-489b-9e68-1f38964d2f63 None admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:36.486873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02df61-0b18-489b-9e68-1f38964d2f63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:36.487184 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02df61-0b18-489b-9e68-1f38964d2f63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:36.488585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02df61-0b18-489b-9e68-1f38964d2f63 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:36.497444 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 637/1277] 10.222.0.64 () {56 vars in 1099 bytes} [Thu Jul 14 17:41:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:36.506109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf58f08-2f08-48fa-9760-25d8881198ce None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:36.603724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf58f08-2f08-48fa-9760-25d8881198ce None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:36.636616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfb925-dc02-4376-b886-6a12338ad911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:36.637399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfb925-dc02-4376-b886-6a12338ad911 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:36.637683 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfb925-dc02-4376-b886-6a12338ad911 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:36.637958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfb925-dc02-4376-b886-6a12338ad911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:36.648706 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfb925-dc02-4376-b886-6a12338ad911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:36.650136 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfb925-dc02-4376-b886-6a12338ad911 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-14cfb925-dc02-4376-b886-6a12338ad911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:41:36.667331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:36.668457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:36.669566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:41:36.669566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:41:36.669566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:36.669566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:36.669566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 641/1278] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:36.674504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02df61-0b18-489b-9e68-1f38964d2f63 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:36.774305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02df61-0b18-489b-9e68-1f38964d2f63 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:36.812897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c44d8-5bfd-42e9-83a1-0608604350da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:36.813566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c44d8-5bfd-42e9-83a1-0608604350da None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:36.813871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c44d8-5bfd-42e9-83a1-0608604350da None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:36.814128 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c44d8-5bfd-42e9-83a1-0608604350da None admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:36.814852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c44d8-5bfd-42e9-83a1-0608604350da None admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:36.815164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c44d8-5bfd-42e9-83a1-0608604350da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:36.816571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c44d8-5bfd-42e9-83a1-0608604350da None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:36.828886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 638/1279] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:36.835953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfb925-dc02-4376-b886-6a12338ad911 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:36.867652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:36.935855 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfb925-dc02-4376-b886-6a12338ad911 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:36.968527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59627a7-aaba-4d1f-a8cd-e0860d6567dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:36.969359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59627a7-aaba-4d1f-a8cd-e0860d6567dc None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:36.969643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59627a7-aaba-4d1f-a8cd-e0860d6567dc None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:36.969921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59627a7-aaba-4d1f-a8cd-e0860d6567dc None admin] PATH_INFO: `/v3/users/placement` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:36.978742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59627a7-aaba-4d1f-a8cd-e0860d6567dc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:36.980262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59627a7-aaba-4d1f-a8cd-e0860d6567dc None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-c59627a7-aaba-4d1f-a8cd-e0860d6567dc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 17:41:36.994361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:36.995486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:36.996739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:36.996739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:36.996739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 17:41:36.996739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 17:41:36.996739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 17:41:36.996739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 17:41:36.996739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 17:41:36.996739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 17:41:36.996739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 17:41:36.996739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:36.996739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 642/1280] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:37.003577 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c44d8-5bfd-42e9-83a1-0608604350da None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:37.007432 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:37.101152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c44d8-5bfd-42e9-83a1-0608604350da None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:37.135459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fba183-ccb9-43e3-932c-21a6db05717d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:37.136169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fba183-ccb9-43e3-932c-21a6db05717d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:37.136454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fba183-ccb9-43e3-932c-21a6db05717d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:37.136730 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fba183-ccb9-43e3-932c-21a6db05717d None admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:37.137564 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fba183-ccb9-43e3-932c-21a6db05717d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:37.137876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fba183-ccb9-43e3-932c-21a6db05717d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:37.139297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fba183-ccb9-43e3-932c-21a6db05717d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:37.156870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 639/1281] 10.222.0.64 () {56 vars in 1143 bytes} [Thu Jul 14 17:41:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:37.162649 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59627a7-aaba-4d1f-a8cd-e0860d6567dc None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:37.250517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:37.260612 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59627a7-aaba-4d1f-a8cd-e0860d6567dc None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:37.293372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc1496-33d8-4b49-8946-72efec90b0d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:37.294019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc1496-33d8-4b49-8946-72efec90b0d9 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:37.294305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc1496-33d8-4b49-8946-72efec90b0d9 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:37.294600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc1496-33d8-4b49-8946-72efec90b0d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:37.360058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc1496-33d8-4b49-8946-72efec90b0d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:37.361516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc1496-33d8-4b49-8946-72efec90b0d9 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:37.450541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:37.460699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 643/1282] 10.222.0.64 () {58 vars in 1282 bytes} [Thu Jul 14 17:41:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 17:41:37.466585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fba183-ccb9-43e3-932c-21a6db05717d None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:37.529934 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:37.568026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fba183-ccb9-43e3-932c-21a6db05717d None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:37.601390 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59ec0b-d376-4b03-8562-d87b4d7cf732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:37.602175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59ec0b-d376-4b03-8562-d87b4d7cf732 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:37.602452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59ec0b-d376-4b03-8562-d87b4d7cf732 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:37.602728 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59ec0b-d376-4b03-8562-d87b4d7cf732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:37.612713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59ec0b-d376-4b03-8562-d87b4d7cf732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:37.614192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59ec0b-d376-4b03-8562-d87b4d7cf732 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-3a59ec0b-d376-4b03-8562-d87b4d7cf732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 17:41:37.632663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:37.633880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:37.635283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 17:41:37.635283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 17:41:37.635283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 17:41:37.635283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:37.635283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 640/1283] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:41:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:37.640056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc1496-33d8-4b49-8946-72efec90b0d9 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:37.736608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc1496-33d8-4b49-8946-72efec90b0d9 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:37.777060 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3520d1e-db0f-4af3-bf38-af959200957c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:37.777734 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3520d1e-db0f-4af3-bf38-af959200957c None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:37.778015 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3520d1e-db0f-4af3-bf38-af959200957c None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:37.778289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3520d1e-db0f-4af3-bf38-af959200957c None admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:37.779016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3520d1e-db0f-4af3-bf38-af959200957c None admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:37.779324 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3520d1e-db0f-4af3-bf38-af959200957c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:37.780714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3520d1e-db0f-4af3-bf38-af959200957c None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:37.792842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 644/1284] 10.222.0.64 () {56 vars in 1109 bytes} [Thu Jul 14 17:41:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:37.798478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59ec0b-d376-4b03-8562-d87b4d7cf732 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:37.897938 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59ec0b-d376-4b03-8562-d87b4d7cf732 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:37.911911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:37.935154 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0d020-4e58-45da-b64e-751a9830f350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:37.935989 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0d020-4e58-45da-b64e-751a9830f350 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:37.936280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0d020-4e58-45da-b64e-751a9830f350 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:37.936561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede0d020-4e58-45da-b64e-751a9830f350 None admin] PATH_INFO: `/v3/projects/service` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:37.943349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0d020-4e58-45da-b64e-751a9830f350 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:37.944926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede0d020-4e58-45da-b64e-751a9830f350 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application [None req-ede0d020-4e58-45da-b64e-751a9830f350 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 17:41:37.958802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:37.959901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:37.960882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 17:41:37.960882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 17:41:37.960882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 17:41:37.960882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 17:41:37.960882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 17:41:37.960882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 17:41:37.960882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 17:41:37.960882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: ERROR keystone.server.flask.application Jul 14 17:41:37.960882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 641/1285] 10.222.0.64 () {56 vars in 1111 bytes} [Thu Jul 14 17:41:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:37.966848 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3520d1e-db0f-4af3-bf38-af959200957c None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:38.031991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:38.088137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3520d1e-db0f-4af3-bf38-af959200957c None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:38.128217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09473e47-fc6b-4a0a-b98d-b3d7679ee2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:38.128895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09473e47-fc6b-4a0a-b98d-b3d7679ee2fd None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:38.129183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09473e47-fc6b-4a0a-b98d-b3d7679ee2fd None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:38.129465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09473e47-fc6b-4a0a-b98d-b3d7679ee2fd None admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:38.130272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09473e47-fc6b-4a0a-b98d-b3d7679ee2fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:38.130588 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09473e47-fc6b-4a0a-b98d-b3d7679ee2fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:38.132157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09473e47-fc6b-4a0a-b98d-b3d7679ee2fd None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:38.146768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 645/1286] 10.222.0.64 () {56 vars in 1148 bytes} [Thu Jul 14 17:41:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:41:38.153569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0d020-4e58-45da-b64e-751a9830f350 None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:38.262362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede0d020-4e58-45da-b64e-751a9830f350 None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:38.296464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b9637e-4e8e-4049-84a9-82d9597b247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:38.297043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9637e-4e8e-4049-84a9-82d9597b247b None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:38.297326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9637e-4e8e-4049-84a9-82d9597b247b None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:38.297603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b9637e-4e8e-4049-84a9-82d9597b247b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:38.298430 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9637e-4e8e-4049-84a9-82d9597b247b None admin] RBAC: Adding query filter params (role.id=c546ef95fd8241208d9fc75ab478ff7d, scope.project.id=de62d2fb17f846f2bce5bef108353fd8, user.id=619e3bec841343c5ad144f1d4e4d436d) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:38.298744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9637e-4e8e-4049-84a9-82d9597b247b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:38.300208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b9637e-4e8e-4049-84a9-82d9597b247b None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:38.306159 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 642/1287] 10.222.0.64 () {56 vars in 1374 bytes} [Thu Jul 14 17:41:38 2022] GET /identity/v3/role_assignments?user.id=619e3bec841343c5ad144f1d4e4d436d&scope.project.id=de62d2fb17f846f2bce5bef108353fd8&role.id=c546ef95fd8241208d9fc75ab478ff7d => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:40.374169 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651d7ac-561a-4756-a10d-9b175d9cbc13 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:40.374670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651d7ac-561a-4756-a10d-9b175d9cbc13 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:40.374670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651d7ac-561a-4756-a10d-9b175d9cbc13 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:40.376050 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 646/1288] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:41:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:41:40.382487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb9a20-0fca-4f00-9d93-1937ba4fcce6 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:40.383031 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb9a20-0fca-4f00-9d93-1937ba4fcce6 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:40.383031 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb9a20-0fca-4f00-9d93-1937ba4fcce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:40.400456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-effb9a20-0fca-4f00-9d93-1937ba4fcce6 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:41:40.401645 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-effb9a20-0fca-4f00-9d93-1937ba4fcce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:41:40.431062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:40.472985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 643/1289] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:41:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:41:40.495700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651d7ac-561a-4756-a10d-9b175d9cbc13 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:40.595083 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651d7ac-561a-4756-a10d-9b175d9cbc13 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:40.606608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:40.630349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4557da8-0163-4e30-adb5-4efad6c22550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:40.631064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4557da8-0163-4e30-adb5-4efad6c22550 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:40.631343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4557da8-0163-4e30-adb5-4efad6c22550 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:40.631613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4557da8-0163-4e30-adb5-4efad6c22550 None admin] PATH_INFO: `/v3/services/placement` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:40.632921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4557da8-0163-4e30-adb5-4efad6c22550 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:40.634274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4557da8-0163-4e30-adb5-4efad6c22550 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-b4557da8-0163-4e30-adb5-4efad6c22550 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:40.646202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:40.647333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:40.648261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 647/1290] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Jul 14 17:41:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:40.653667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb9a20-0fca-4f00-9d93-1937ba4fcce6 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:40.751145 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb9a20-0fca-4f00-9d93-1937ba4fcce6 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:40.791058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a7e7e9-aff0-4991-a49b-2926b55de866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:40.791732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7e7e9-aff0-4991-a49b-2926b55de866 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:40.792046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7e7e9-aff0-4991-a49b-2926b55de866 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:40.792323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a7e7e9-aff0-4991-a49b-2926b55de866 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:40.793030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7e7e9-aff0-4991-a49b-2926b55de866 None admin] RBAC: Adding query filter params (name=placement) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:40.793372 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7e7e9-aff0-4991-a49b-2926b55de866 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:40.794700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a7e7e9-aff0-4991-a49b-2926b55de866 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:40.799748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 644/1291] 10.222.0.64 () {56 vars in 1116 bytes} [Thu Jul 14 17:41:40 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:40.807233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4557da8-0163-4e30-adb5-4efad6c22550 None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:40.912528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4557da8-0163-4e30-adb5-4efad6c22550 None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:40.945191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa8c5d-9f6d-4402-8d56-a9c3f3ae24c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:40.945852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa8c5d-9f6d-4402-8d56-a9c3f3ae24c9 None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:40.946138 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa8c5d-9f6d-4402-8d56-a9c3f3ae24c9 None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:40.946417 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa8c5d-9f6d-4402-8d56-a9c3f3ae24c9 None admin] PATH_INFO: `/v3/services` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:40.947134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa8c5d-9f6d-4402-8d56-a9c3f3ae24c9 None admin] RBAC: Adding query filter params (type=placement) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:40.947448 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa8c5d-9f6d-4402-8d56-a9c3f3ae24c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:40.948847 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa8c5d-9f6d-4402-8d56-a9c3f3ae24c9 None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:40.953376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 648/1292] 10.222.0.64 () {56 vars in 1116 bytes} [Thu Jul 14 17:41:40 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:42.990409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64938b-ce00-485c-a301-74ae5e1cf124 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:42.990945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64938b-ce00-485c-a301-74ae5e1cf124 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:42.990945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa64938b-ce00-485c-a301-74ae5e1cf124 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:42.992869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 645/1293] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:41:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:41:42.998020 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5ea7c-93de-4cfd-a5e2-59e266960556 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:42.998422 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5ea7c-93de-4cfd-a5e2-59e266960556 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:42.998565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5ea7c-93de-4cfd-a5e2-59e266960556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:43.015754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-f8c5ea7c-93de-4cfd-a5e2-59e266960556 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:41:43.016956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f8c5ea7c-93de-4cfd-a5e2-59e266960556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:41:43.045253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:43.081276 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 649/1294] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:41:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:41:43.104213 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64938b-ce00-485c-a301-74ae5e1cf124 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:43.135897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:43.213275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa64938b-ce00-485c-a301-74ae5e1cf124 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:43.247527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f31e5-d6c7-497f-82e1-2e064d37f691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:43.248270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f31e5-d6c7-497f-82e1-2e064d37f691 None admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:43.248556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f31e5-d6c7-497f-82e1-2e064d37f691 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:43.248832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f31e5-d6c7-497f-82e1-2e064d37f691 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:43.249554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f31e5-d6c7-497f-82e1-2e064d37f691 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:43.250912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f31e5-d6c7-497f-82e1-2e064d37f691 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:43.265129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 646/1295] 10.222.0.64 () {60 vars in 1141 bytes} [Thu Jul 14 17:41:43 2022] POST /identity/v3/services => generated 246 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:41:45.320518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b8732-e027-43d7-b0a3-03d52b6f3adf None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:45.321074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b8732-e027-43d7-b0a3-03d52b6f3adf None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:45.321074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2b8732-e027-43d7-b0a3-03d52b6f3adf None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:45.322452 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 650/1296] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:41:45.327685 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97eb2be-531d-44a1-bb79-356726c8d98a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:45.328097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97eb2be-531d-44a1-bb79-356726c8d98a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:45.328274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97eb2be-531d-44a1-bb79-356726c8d98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:45.344676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-e97eb2be-531d-44a1-bb79-356726c8d98a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:41:45.345871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e97eb2be-531d-44a1-bb79-356726c8d98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:41:45.417643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 647/1297] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:41:45 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 17:41:45.442028 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b8732-e027-43d7-b0a3-03d52b6f3adf None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:45.541381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2b8732-e027-43d7-b0a3-03d52b6f3adf None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:45.578815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb8930-99c1-4cc8-b6c2-49adb683d71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:45.579610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb8930-99c1-4cc8-b6c2-49adb683d71a None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:45.579896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb8930-99c1-4cc8-b6c2-49adb683d71a None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:45.580214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bb8930-99c1-4cc8-b6c2-49adb683d71a None admin] PATH_INFO: `/v3/services/placement` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:45.580869 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb8930-99c1-4cc8-b6c2-49adb683d71a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:45.582244 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bb8930-99c1-4cc8-b6c2-49adb683d71a None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-25bb8930-99c1-4cc8-b6c2-49adb683d71a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:45.593942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:45.595104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:45.596286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 651/1298] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Jul 14 17:41:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:45.601801 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97eb2be-531d-44a1-bb79-356726c8d98a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:45.620280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:45.701878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97eb2be-531d-44a1-bb79-356726c8d98a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:45.736869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a6637-b071-4cd7-b63a-4389b5839dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:45.737540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a6637-b071-4cd7-b63a-4389b5839dff None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:45.737822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a6637-b071-4cd7-b63a-4389b5839dff None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:45.738106 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a6637-b071-4cd7-b63a-4389b5839dff None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:45.738812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a6637-b071-4cd7-b63a-4389b5839dff None admin] RBAC: Adding query filter params (name=placement) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:45.739121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a6637-b071-4cd7-b63a-4389b5839dff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:45.740545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26a6637-b071-4cd7-b63a-4389b5839dff None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:45.745765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 648/1299] 10.222.0.64 () {56 vars in 1116 bytes} [Thu Jul 14 17:41:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:45.751844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb8930-99c1-4cc8-b6c2-49adb683d71a None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:45.787778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:45.865153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bb8930-99c1-4cc8-b6c2-49adb683d71a None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:45.898637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42193463-c9cb-4a51-9383-6d823c45271d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:45.899290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42193463-c9cb-4a51-9383-6d823c45271d None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:45.899572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42193463-c9cb-4a51-9383-6d823c45271d None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:45.899842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42193463-c9cb-4a51-9383-6d823c45271d None admin] PATH_INFO: `/v3/endpoints` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:45.900863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42193463-c9cb-4a51-9383-6d823c45271d None admin] RBAC: Adding query filter params (interface=public, service_id=7de7caacec744ec489ebd4b60eac7f55, region_id=RegionOne) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:45.901180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42193463-c9cb-4a51-9383-6d823c45271d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:45.902657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42193463-c9cb-4a51-9383-6d823c45271d None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:45.910018 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 652/1300] 10.222.0.64 () {56 vars in 1251 bytes} [Thu Jul 14 17:41:45 2022] GET /identity/v3/endpoints?service_id=7de7caacec744ec489ebd4b60eac7f55&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:45.915938 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a6637-b071-4cd7-b63a-4389b5839dff None admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:45.997513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:46.017320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26a6637-b071-4cd7-b63a-4389b5839dff None admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:46.051236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a691030-5fd4-4ff5-acb3-6166974b1768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:46.051903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a691030-5fd4-4ff5-acb3-6166974b1768 None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:46.052221 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a691030-5fd4-4ff5-acb3-6166974b1768 None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:46.052506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a691030-5fd4-4ff5-acb3-6166974b1768 None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:46.053190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a691030-5fd4-4ff5-acb3-6166974b1768 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:46.054611 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a691030-5fd4-4ff5-acb3-6166974b1768 None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:46.061432 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 649/1301] 10.222.0.64 () {56 vars in 1087 bytes} [Thu Jul 14 17:41:45 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:41:48.092844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede4ac-8f7e-4d8f-b0d2-e9e202ba94b4 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:48.093421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede4ac-8f7e-4d8f-b0d2-e9e202ba94b4 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:48.093421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede4ac-8f7e-4d8f-b0d2-e9e202ba94b4 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:48.094935 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 653/1302] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:41:48.099962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4683ce-7e14-4a79-8029-55c114932a53 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:48.100367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4683ce-7e14-4a79-8029-55c114932a53 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:48.100525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4683ce-7e14-4a79-8029-55c114932a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:48.117484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-2c4683ce-7e14-4a79-8029-55c114932a53 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:41:48.118674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-2c4683ce-7e14-4a79-8029-55c114932a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:41:48.183673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 650/1303] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:41:48 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 17:41:48.206265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edede4ac-8f7e-4d8f-b0d2-e9e202ba94b4 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:48.300021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:48.307082 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edede4ac-8f7e-4d8f-b0d2-e9e202ba94b4 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:48.340144 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5875f7-28ad-49af-869d-a95ea72709cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:48.340940 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5875f7-28ad-49af-869d-a95ea72709cc None admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:48.341218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5875f7-28ad-49af-869d-a95ea72709cc None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:48.341490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5875f7-28ad-49af-869d-a95ea72709cc None admin] PATH_INFO: `/v3/services/placement` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:48.342139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5875f7-28ad-49af-869d-a95ea72709cc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:48.343552 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5875f7-28ad-49af-869d-a95ea72709cc None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application [None req-af5875f7-28ad-49af-869d-a95ea72709cc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 17:41:48.355348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 17:41:48.356581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application with Lock( Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self._enter() Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application return self.creator() Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application created_value = creator( Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: ERROR keystone.server.flask.application Jul 14 17:41:48.357787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 654/1304] 10.222.0.64 () {56 vars in 1117 bytes} [Thu Jul 14 17:41:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 17:41:48.363481 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4683ce-7e14-4a79-8029-55c114932a53 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:48.439049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:48.463669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4683ce-7e14-4a79-8029-55c114932a53 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:48.497598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181b486-bab3-48d5-9474-51ee569e413d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:48.498268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181b486-bab3-48d5-9474-51ee569e413d None admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:48.498557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181b486-bab3-48d5-9474-51ee569e413d None admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:48.498836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181b486-bab3-48d5-9474-51ee569e413d None admin] PATH_INFO: `/v3/services` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:48.499544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181b486-bab3-48d5-9474-51ee569e413d None admin] RBAC: Adding query filter params (name=placement) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:41:48.499854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181b486-bab3-48d5-9474-51ee569e413d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:48.501250 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181b486-bab3-48d5-9474-51ee569e413d None admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:48.506517 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 651/1305] 10.222.0.64 () {56 vars in 1116 bytes} [Thu Jul 14 17:41:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:41:48.513774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5875f7-28ad-49af-869d-a95ea72709cc None admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:41:48.612196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5875f7-28ad-49af-869d-a95ea72709cc None admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:41:48.646070 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ca530-4c7e-4e8d-b5c8-fdba97d5032b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:41:48.646722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ca530-4c7e-4e8d-b5c8-fdba97d5032b None admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:41:48.646999 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ca530-4c7e-4e8d-b5c8-fdba97d5032b None admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:41:48.647266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ca530-4c7e-4e8d-b5c8-fdba97d5032b None admin] PATH_INFO: `/v3/endpoints` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:41:48.648069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ca530-4c7e-4e8d-b5c8-fdba97d5032b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:41:48.649924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662ca530-4c7e-4e8d-b5c8-fdba97d5032b None admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:41:48.668348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 655/1306] 10.222.0.64 () {60 vars in 1144 bytes} [Thu Jul 14 17:41:48 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:41:58.503582 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=30 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:41:58.664139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:42:53.013919 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36773717-e262-4c56-9ea5-9f934139adc5 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:42:53.014385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36773717-e262-4c56-9ea5-9f934139adc5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:42:53.014608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36773717-e262-4c56-9ea5-9f934139adc5 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:42:53.020337 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 652/1307] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:42:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:42:53.022038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af3b40-e9ec-4f1a-88e5-a96ac92b7542 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:42:53.022481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af3b40-e9ec-4f1a-88e5-a96ac92b7542 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:42:53.022758 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af3b40-e9ec-4f1a-88e5-a96ac92b7542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:42:53.190149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-c6af3b40-e9ec-4f1a-88e5-a96ac92b7542 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:42:53.193687 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c6af3b40-e9ec-4f1a-88e5-a96ac92b7542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:42:53.372141 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 656/1308] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:42:53.422332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a144be-f470-48c5-9d43-a4170d61abdc None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:42:53.422581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a144be-f470-48c5-9d43-a4170d61abdc None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:42:53.422867 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a144be-f470-48c5-9d43-a4170d61abdc None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:42:53.424629 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 653/1309] 10.222.0.64 () {54 vars in 964 bytes} [Thu Jul 14 17:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:42:53.432584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ba687-5294-49a1-8ffe-87aeef298e17 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:42:53.433437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ba687-5294-49a1-8ffe-87aeef298e17 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:42:53.434176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ba687-5294-49a1-8ffe-87aeef298e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:42:53.540435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-9a7ba687-5294-49a1-8ffe-87aeef298e17 None None] MFA Rules not processed for user `a5adec9924444e29b8964f7969452592`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:42:53.541890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9a7ba687-5294-49a1-8ffe-87aeef298e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:42:53.569646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:42:53.638959 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 657/1310] 10.222.0.64 () {58 vars in 1061 bytes} [Thu Jul 14 17:42:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:42:53.648253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a144be-f470-48c5-9d43-a4170d61abdc None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:42:53.650308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-64a144be-f470-48c5-9d43-a4170d61abdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:42:53.780266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a144be-f470-48c5-9d43-a4170d61abdc None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:42:53.867240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16abf2a-c961-46d6-9f62-fdf3da793d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:42:53.868209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16abf2a-c961-46d6-9f62-fdf3da793d76 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:42:53.868646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16abf2a-c961-46d6-9f62-fdf3da793d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:42:53.869103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16abf2a-c961-46d6-9f62-fdf3da793d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:42:53.872167 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f16abf2a-c961-46d6-9f62-fdf3da793d76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:42:53.937395 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:42:54.012835 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16abf2a-c961-46d6-9f62-fdf3da793d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:42:54.014635 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16abf2a-c961-46d6-9f62-fdf3da793d76 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:42:54.040167 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 654/1311] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:42:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:42:56.449758 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61254b-9c66-41e6-8df9-7674e5a0c771 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:42:56.450274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61254b-9c66-41e6-8df9-7674e5a0c771 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:42:56.450365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce61254b-9c66-41e6-8df9-7674e5a0c771 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:42:56.451864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 658/1312] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:42:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:42:56.456925 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28268dc6-9c04-44b5-8796-3807783d3e71 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:42:56.457363 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28268dc6-9c04-44b5-8796-3807783d3e71 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:42:56.457617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28268dc6-9c04-44b5-8796-3807783d3e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:42:56.530477 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-28268dc6-9c04-44b5-8796-3807783d3e71 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:42:56.531719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-28268dc6-9c04-44b5-8796-3807783d3e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:42:56.634503 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 655/1313] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:42:56.673859 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61254b-9c66-41e6-8df9-7674e5a0c771 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:42:56.675392 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ce61254b-9c66-41e6-8df9-7674e5a0c771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:42:56.785844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce61254b-9c66-41e6-8df9-7674e5a0c771 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:42:56.860562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:42:56.863202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad21388-5e6e-4870-8ce3-62f345eb5fce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:42:56.864114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad21388-5e6e-4870-8ce3-62f345eb5fce service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:42:56.865306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad21388-5e6e-4870-8ce3-62f345eb5fce service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:42:56.865306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad21388-5e6e-4870-8ce3-62f345eb5fce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:42:56.867386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-bad21388-5e6e-4870-8ce3-62f345eb5fce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:42:56.988645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad21388-5e6e-4870-8ce3-62f345eb5fce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:42:56.990097 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad21388-5e6e-4870-8ce3-62f345eb5fce service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:42:57.009684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 659/1314] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:42:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:42:59.379378 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79bea3-daa6-46f0-86b5-38f24fad951f None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:42:59.379898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79bea3-daa6-46f0-86b5-38f24fad951f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:42:59.379898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b79bea3-daa6-46f0-86b5-38f24fad951f None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:42:59.381252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 656/1315] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:42:59.386392 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555f3f2-82ba-41a9-abdb-4243357ed568 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:42:59.386800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555f3f2-82ba-41a9-abdb-4243357ed568 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:42:59.386964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555f3f2-82ba-41a9-abdb-4243357ed568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:42:59.450442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5555f3f2-82ba-41a9-abdb-4243357ed568 None None] MFA Rules not processed for user `7db339453aae4a3fa6c2ae5cb3a3a2eb`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:42:59.451625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5555f3f2-82ba-41a9-abdb-4243357ed568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:42:59.505967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:42:59.550840 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 660/1316] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:42:59 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:42:59.590403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79bea3-daa6-46f0-86b5-38f24fad951f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:42:59.591956 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4b79bea3-daa6-46f0-86b5-38f24fad951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:42:59.601665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:42:59.702066 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b79bea3-daa6-46f0-86b5-38f24fad951f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:42:59.787751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245eea0-32e4-47a2-b58f-e415b1142f19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:42:59.788450 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245eea0-32e4-47a2-b58f-e415b1142f19 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:42:59.788738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245eea0-32e4-47a2-b58f-e415b1142f19 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:42:59.789017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245eea0-32e4-47a2-b58f-e415b1142f19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:42:59.790763 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-2245eea0-32e4-47a2-b58f-e415b1142f19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:42:59.857804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:42:59.909581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245eea0-32e4-47a2-b58f-e415b1142f19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:42:59.911046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2245eea0-32e4-47a2-b58f-e415b1142f19 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:42:59.932828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 657/1317] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:42:59 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:43:02.588502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837983c-531c-45ba-9a7d-cc285e997fba None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:02.589062 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837983c-531c-45ba-9a7d-cc285e997fba None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:02.589062 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2837983c-531c-45ba-9a7d-cc285e997fba None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:02.590466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 661/1318] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:02.595808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a777c2-7184-43ea-834d-0bb29a6ff7e6 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:02.596243 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a777c2-7184-43ea-834d-0bb29a6ff7e6 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:02.596440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a777c2-7184-43ea-834d-0bb29a6ff7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:02.661012 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f6a777c2-7184-43ea-834d-0bb29a6ff7e6 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:02.662237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f6a777c2-7184-43ea-834d-0bb29a6ff7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:02.757499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 658/1319] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:43:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:02.814375 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a74cf8-5335-485c-af67-b07440ef874e None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:02.814663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a74cf8-5335-485c-af67-b07440ef874e None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:02.814946 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a74cf8-5335-485c-af67-b07440ef874e None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:02.816409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 662/1320] 10.222.0.64 () {54 vars in 964 bytes} [Thu Jul 14 17:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:02.823049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db25326-e38b-4cb1-b2c5-ffc0e2dcb65b None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:02.823344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db25326-e38b-4cb1-b2c5-ffc0e2dcb65b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:02.823627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db25326-e38b-4cb1-b2c5-ffc0e2dcb65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:02.844980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:02.907783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-6db25326-e38b-4cb1-b2c5-ffc0e2dcb65b None None] MFA Rules not processed for user `a5adec9924444e29b8964f7969452592`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:02.909530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6db25326-e38b-4cb1-b2c5-ffc0e2dcb65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:02.987287 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 659/1321] 10.222.0.64 () {58 vars in 1061 bytes} [Thu Jul 14 17:43:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:02.996379 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a74cf8-5335-485c-af67-b07440ef874e None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:43:02.998436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-25a74cf8-5335-485c-af67-b07440ef874e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:03.101063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a74cf8-5335-485c-af67-b07440ef874e None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:43:03.180106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee629553-5f05-4031-8a24-a1728ebeb99a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:43:03.180760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee629553-5f05-4031-8a24-a1728ebeb99a service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:03.181048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee629553-5f05-4031-8a24-a1728ebeb99a service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:03.181328 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee629553-5f05-4031-8a24-a1728ebeb99a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:03.183036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ee629553-5f05-4031-8a24-a1728ebeb99a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:03.191161 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:03.291849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee629553-5f05-4031-8a24-a1728ebeb99a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:43:03.293820 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee629553-5f05-4031-8a24-a1728ebeb99a service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:43:03.311882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 663/1322] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:43:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:43:06.513760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d10b4e-da7a-4661-859c-ed2e29eaef9d None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:06.514308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d10b4e-da7a-4661-859c-ed2e29eaef9d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:06.514308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d10b4e-da7a-4661-859c-ed2e29eaef9d None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:06.515704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 660/1323] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:06.521614 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0be102-0dbc-4a2c-b60f-855ca0201e9f None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:06.522309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0be102-0dbc-4a2c-b60f-855ca0201e9f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:06.522309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0be102-0dbc-4a2c-b60f-855ca0201e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:06.595757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-ee0be102-0dbc-4a2c-b60f-855ca0201e9f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:06.597088 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ee0be102-0dbc-4a2c-b60f-855ca0201e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:06.669938 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:06.716249 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 664/1324] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:43:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:06.754203 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d10b4e-da7a-4661-859c-ed2e29eaef9d None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:43:06.755757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-71d10b4e-da7a-4661-859c-ed2e29eaef9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:06.867398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d10b4e-da7a-4661-859c-ed2e29eaef9d None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:43:06.882591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:06.959898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d903cb46-3fd8-4124-999f-c9543a1fd234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:43:06.960594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903cb46-3fd8-4124-999f-c9543a1fd234 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:06.960883 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903cb46-3fd8-4124-999f-c9543a1fd234 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:06.961159 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d903cb46-3fd8-4124-999f-c9543a1fd234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:06.962874 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d903cb46-3fd8-4124-999f-c9543a1fd234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:07.117659 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903cb46-3fd8-4124-999f-c9543a1fd234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:43:07.119494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d903cb46-3fd8-4124-999f-c9543a1fd234 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:43:07.140400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 661/1325] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:43:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:43:09.301760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0940168-39ea-492e-860a-792ad55721d1 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:09.302426 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0940168-39ea-492e-860a-792ad55721d1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:09.302426 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0940168-39ea-492e-860a-792ad55721d1 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:09.303876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 665/1326] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:09.309729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4fc4d-51ea-40c9-ac71-1feea7d4830f None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:09.310356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4fc4d-51ea-40c9-ac71-1feea7d4830f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:09.310356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4fc4d-51ea-40c9-ac71-1feea7d4830f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:09.374227 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:09.408172 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-6eb4fc4d-51ea-40c9-ac71-1feea7d4830f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:09.410242 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6eb4fc4d-51ea-40c9-ac71-1feea7d4830f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:09.594623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 662/1327] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:43:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:09.637092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0940168-39ea-492e-860a-792ad55721d1 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:43:09.640506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d0940168-39ea-492e-860a-792ad55721d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:09.806124 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0940168-39ea-492e-860a-792ad55721d1 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:43:09.919152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dbb37-02ce-442a-a02e-e770ca75d886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:43:09.920214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dbb37-02ce-442a-a02e-e770ca75d886 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:09.920715 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dbb37-02ce-442a-a02e-e770ca75d886 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:09.921187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dbb37-02ce-442a-a02e-e770ca75d886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:09.924165 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-838dbb37-02ce-442a-a02e-e770ca75d886 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:09.946123 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:10.071555 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838dbb37-02ce-442a-a02e-e770ca75d886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:43:10.074320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838dbb37-02ce-442a-a02e-e770ca75d886 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:43:10.106712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 666/1328] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:43:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:43:12.429744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817ea85-f418-449a-86cf-d4a599ac888c None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:12.430357 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817ea85-f418-449a-86cf-d4a599ac888c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:12.430357 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817ea85-f418-449a-86cf-d4a599ac888c None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:12.431877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 663/1329] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:12.438757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f9f6e-228e-4024-9c89-f69843995cd1 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:12.439350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f9f6e-228e-4024-9c89-f69843995cd1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:12.439523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8f9f6e-228e-4024-9c89-f69843995cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:12.534005 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-fd8f9f6e-228e-4024-9c89-f69843995cd1 None None] MFA Rules not processed for user `7db339453aae4a3fa6c2ae5cb3a3a2eb`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:12.535554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-fd8f9f6e-228e-4024-9c89-f69843995cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:12.613386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:12.669329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 667/1330] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:43:12 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:12.710282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817ea85-f418-449a-86cf-d4a599ac888c None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:43:12.713022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3817ea85-f418-449a-86cf-d4a599ac888c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:12.913012 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817ea85-f418-449a-86cf-d4a599ac888c None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:43:12.951619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:13.018999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be22e4d0-4fdb-483b-9c7f-c0c3a02c8c03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:43:13.019702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22e4d0-4fdb-483b-9c7f-c0c3a02c8c03 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:13.019957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22e4d0-4fdb-483b-9c7f-c0c3a02c8c03 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:13.020237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be22e4d0-4fdb-483b-9c7f-c0c3a02c8c03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:13.021913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-be22e4d0-4fdb-483b-9c7f-c0c3a02c8c03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:13.175061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22e4d0-4fdb-483b-9c7f-c0c3a02c8c03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:43:13.176589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be22e4d0-4fdb-483b-9c7f-c0c3a02c8c03 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:43:13.204306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 664/1331] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:43:12 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:43:15.778359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8197e0-7b0f-4f8a-92ac-70d270a0aa80 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:15.778945 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8197e0-7b0f-4f8a-92ac-70d270a0aa80 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:15.779376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8197e0-7b0f-4f8a-92ac-70d270a0aa80 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:15.781478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 668/1332] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:43:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:15.787464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585aa23-057b-4b12-9c4a-3240c89a7a6d None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:15.788045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585aa23-057b-4b12-9c4a-3240c89a7a6d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:15.788045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585aa23-057b-4b12-9c4a-3240c89a7a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:15.870053 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:15.876622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-c585aa23-057b-4b12-9c4a-3240c89a7a6d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:15.878442 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c585aa23-057b-4b12-9c4a-3240c89a7a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:15.999346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 665/1333] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:16.038330 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8197e0-7b0f-4f8a-92ac-70d270a0aa80 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:43:16.039996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5a8197e0-7b0f-4f8a-92ac-70d270a0aa80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:16.189630 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8197e0-7b0f-4f8a-92ac-70d270a0aa80 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:43:16.295515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88f91c0-34ba-40b8-9285-2942594bca98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:43:16.296187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f91c0-34ba-40b8-9285-2942594bca98 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:16.296494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f91c0-34ba-40b8-9285-2942594bca98 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:16.296789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88f91c0-34ba-40b8-9285-2942594bca98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:16.298767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c88f91c0-34ba-40b8-9285-2942594bca98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:16.331099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:16.435064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f91c0-34ba-40b8-9285-2942594bca98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:43:16.436807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88f91c0-34ba-40b8-9285-2942594bca98 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:43:16.458468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 669/1334] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:43:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:43:18.821399 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cecbaf-2319-4b7e-ad88-3f43bc0b2227 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:18.822036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cecbaf-2319-4b7e-ad88-3f43bc0b2227 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:18.822036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cecbaf-2319-4b7e-ad88-3f43bc0b2227 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:18.823364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 666/1335] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:18.831315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1d1c-c590-4818-a849-49fc17603149 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:18.832033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1d1c-c590-4818-a849-49fc17603149 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:18.832033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f1d1c-c590-4818-a849-49fc17603149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:18.902366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-aa0f1d1c-c590-4818-a849-49fc17603149 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:18.903686 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-aa0f1d1c-c590-4818-a849-49fc17603149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:18.997221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:19.029205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 670/1336] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:43:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:19.073987 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cecbaf-2319-4b7e-ad88-3f43bc0b2227 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:43:19.075626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d6cecbaf-2319-4b7e-ad88-3f43bc0b2227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:19.191492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cecbaf-2319-4b7e-ad88-3f43bc0b2227 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:43:19.222955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:19.270751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda93d4c-bb93-46fc-a47a-c1b0447c9802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:43:19.271419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda93d4c-bb93-46fc-a47a-c1b0447c9802 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:19.271704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda93d4c-bb93-46fc-a47a-c1b0447c9802 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:19.272049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda93d4c-bb93-46fc-a47a-c1b0447c9802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:19.273697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-dda93d4c-bb93-46fc-a47a-c1b0447c9802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:19.413066 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda93d4c-bb93-46fc-a47a-c1b0447c9802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:43:19.414481 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda93d4c-bb93-46fc-a47a-c1b0447c9802 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:43:19.438261 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 667/1337] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:43:22.156561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268214c7-bb00-42aa-ab2f-9f7965ca1148 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:22.157157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268214c7-bb00-42aa-ab2f-9f7965ca1148 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:22.157157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268214c7-bb00-42aa-ab2f-9f7965ca1148 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:22.158591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 671/1338] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:22.164190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1f526-e3a7-4b5a-bd28-1ebe10330c02 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:22.164852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1f526-e3a7-4b5a-bd28-1ebe10330c02 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:22.164852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1f526-e3a7-4b5a-bd28-1ebe10330c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:22.238915 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-d4f1f526-e3a7-4b5a-bd28-1ebe10330c02 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:22.241016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d4f1f526-e3a7-4b5a-bd28-1ebe10330c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:22.255815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:22.364518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 668/1339] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:43:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:22.410431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268214c7-bb00-42aa-ab2f-9f7965ca1148 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:43:22.411982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-268214c7-bb00-42aa-ab2f-9f7965ca1148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:22.534673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268214c7-bb00-42aa-ab2f-9f7965ca1148 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:43:22.609668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aec017-e50d-4fda-a27c-2f3ad1701b19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:43:22.610710 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aec017-e50d-4fda-a27c-2f3ad1701b19 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:22.611029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aec017-e50d-4fda-a27c-2f3ad1701b19 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:22.611517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aec017-e50d-4fda-a27c-2f3ad1701b19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:22.614222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e0aec017-e50d-4fda-a27c-2f3ad1701b19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:22.656557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:22.737637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aec017-e50d-4fda-a27c-2f3ad1701b19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:43:22.739085 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aec017-e50d-4fda-a27c-2f3ad1701b19 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:43:22.759890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 672/1340] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:43:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:43:27.152356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5beaa-4aaa-4e92-8191-62dfd8c2a17a None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:27.161079 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5beaa-4aaa-4e92-8191-62dfd8c2a17a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:27.161406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d5beaa-4aaa-4e92-8191-62dfd8c2a17a None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:27.162954 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 669/1341] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:43:27 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:43:27.169308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c56e00-091e-4f58-a5f3-61b24853160d None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:27.169845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c56e00-091e-4f58-a5f3-61b24853160d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:27.170621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c56e00-091e-4f58-a5f3-61b24853160d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:27.266918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-35c56e00-091e-4f58-a5f3-61b24853160d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:27.269081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-35c56e00-091e-4f58-a5f3-61b24853160d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:27.399439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:27.412548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 673/1342] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:43:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:27.453978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5beaa-4aaa-4e92-8191-62dfd8c2a17a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:43:27.455523 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-34d5beaa-4aaa-4e92-8191-62dfd8c2a17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:27.579653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d5beaa-4aaa-4e92-8191-62dfd8c2a17a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:43:27.626834 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:27.666412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3204b267-82c9-4350-9cf2-aa15295db865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:43:27.667071 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204b267-82c9-4350-9cf2-aa15295db865 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:27.667356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204b267-82c9-4350-9cf2-aa15295db865 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:27.667636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3204b267-82c9-4350-9cf2-aa15295db865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:27.669361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3204b267-82c9-4350-9cf2-aa15295db865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:27.806540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204b267-82c9-4350-9cf2-aa15295db865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:43:27.808799 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3204b267-82c9-4350-9cf2-aa15295db865 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:43:27.839601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 670/1343] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:43:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:43:30.908401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b004ea-b35b-44de-9795-842026e770b9 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:30.908968 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b004ea-b35b-44de-9795-842026e770b9 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:30.908968 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b004ea-b35b-44de-9795-842026e770b9 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:30.910360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 674/1344] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:30.916202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539adca-e54d-4498-b21b-445b7c1bc0ea None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:30.916693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539adca-e54d-4498-b21b-445b7c1bc0ea None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:30.916823 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2539adca-e54d-4498-b21b-445b7c1bc0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:31.003009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-2539adca-e54d-4498-b21b-445b7c1bc0ea None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:31.004361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-2539adca-e54d-4498-b21b-445b7c1bc0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:31.024031 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:31.136941 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 671/1345] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:43:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:31.176346 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b004ea-b35b-44de-9795-842026e770b9 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:43:31.177873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-18b004ea-b35b-44de-9795-842026e770b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:31.300626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b004ea-b35b-44de-9795-842026e770b9 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:43:31.383585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76358ce-98cd-423c-b55f-29137d1ea9d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:43:31.384242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76358ce-98cd-423c-b55f-29137d1ea9d6 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:31.384530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76358ce-98cd-423c-b55f-29137d1ea9d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:31.384811 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76358ce-98cd-423c-b55f-29137d1ea9d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:31.386570 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e76358ce-98cd-423c-b55f-29137d1ea9d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:31.438204 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:31.527168 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76358ce-98cd-423c-b55f-29137d1ea9d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:43:31.530983 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76358ce-98cd-423c-b55f-29137d1ea9d6 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:43:31.554823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 675/1346] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:43:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:43:36.326431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecfe74-b56d-4e59-96c3-1bb5fd3049ca None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:36.327442 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecfe74-b56d-4e59-96c3-1bb5fd3049ca None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:36.327538 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecfe74-b56d-4e59-96c3-1bb5fd3049ca None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:36.329079 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 672/1347] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:43:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:36.334827 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc14505-6f8c-4f73-a7a0-70ec3298e5d4 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:36.335312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc14505-6f8c-4f73-a7a0-70ec3298e5d4 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:36.335393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc14505-6f8c-4f73-a7a0-70ec3298e5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:36.404352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5fc14505-6f8c-4f73-a7a0-70ec3298e5d4 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:36.405584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5fc14505-6f8c-4f73-a7a0-70ec3298e5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:36.510042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:36.512840 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 676/1348] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:36.551459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecfe74-b56d-4e59-96c3-1bb5fd3049ca None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:43:36.553074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-56ecfe74-b56d-4e59-96c3-1bb5fd3049ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:36.659154 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecfe74-b56d-4e59-96c3-1bb5fd3049ca None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:43:36.707164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:36.735398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fd9e5-37af-4f08-8fe8-40d06ac76b1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:43:36.736106 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fd9e5-37af-4f08-8fe8-40d06ac76b1e service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:36.736391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fd9e5-37af-4f08-8fe8-40d06ac76b1e service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:36.736669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fd9e5-37af-4f08-8fe8-40d06ac76b1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:36.738383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-714fd9e5-37af-4f08-8fe8-40d06ac76b1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:36.858370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fd9e5-37af-4f08-8fe8-40d06ac76b1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:43:36.859725 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fd9e5-37af-4f08-8fe8-40d06ac76b1e service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:43:36.877729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 673/1349] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:43:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:43:41.934837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaab83-7fd6-422b-8d9a-9f2736e52bde None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:41.935407 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaab83-7fd6-422b-8d9a-9f2736e52bde None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:41.935590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaab83-7fd6-422b-8d9a-9f2736e52bde None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:41.937824 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 677/1350] 10.222.0.64 () {54 vars in 921 bytes} [Thu Jul 14 17:43:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:41.943956 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63054ad7-41d5-4915-979c-e694357b7098 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:41.943956 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63054ad7-41d5-4915-979c-e694357b7098 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:41.943956 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63054ad7-41d5-4915-979c-e694357b7098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:42.071148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-63054ad7-41d5-4915-979c-e694357b7098 None None] MFA Rules not processed for user `619e3bec841343c5ad144f1d4e4d436d`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:42.074279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-63054ad7-41d5-4915-979c-e694357b7098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:42.123215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:42.252398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 674/1351] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Jul 14 17:43:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:46.412735 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd28434-39c0-488c-8022-21cee838f1c1 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:46.413242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd28434-39c0-488c-8022-21cee838f1c1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:46.413242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd28434-39c0-488c-8022-21cee838f1c1 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:46.414761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 678/1352] 10.222.0.64 () {54 vars in 921 bytes} [Thu Jul 14 17:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:46.419664 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90252fcb-d7d2-40a7-856e-5e809dd6e5b3 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:46.420121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90252fcb-d7d2-40a7-856e-5e809dd6e5b3 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:46.420269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90252fcb-d7d2-40a7-856e-5e809dd6e5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:46.503272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-90252fcb-d7d2-40a7-856e-5e809dd6e5b3 None None] MFA Rules not processed for user `619e3bec841343c5ad144f1d4e4d436d`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:46.504541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-90252fcb-d7d2-40a7-856e-5e809dd6e5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:46.581888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:46.620504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 675/1353] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Jul 14 17:43:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:48.102783 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3576c5a-9aef-4e00-9c61-45e7ba627760 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:48.103346 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3576c5a-9aef-4e00-9c61-45e7ba627760 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:48.103346 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3576c5a-9aef-4e00-9c61-45e7ba627760 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:48.104928 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 679/1354] 10.222.0.64 () {54 vars in 921 bytes} [Thu Jul 14 17:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:48.111015 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b4a58-f17a-4de9-95e7-144c6c344850 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:48.111463 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b4a58-f17a-4de9-95e7-144c6c344850 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:48.111806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71b4a58-f17a-4de9-95e7-144c6c344850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:48.201856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-e71b4a58-f17a-4de9-95e7-144c6c344850 None None] MFA Rules not processed for user `619e3bec841343c5ad144f1d4e4d436d`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:48.203167 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e71b4a58-f17a-4de9-95e7-144c6c344850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:48.320587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 676/1355] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Jul 14 17:43:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:51.766069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cee8d-cf93-42fc-9155-8642b12325e0 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:51.767115 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cee8d-cf93-42fc-9155-8642b12325e0 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:51.767635 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049cee8d-cf93-42fc-9155-8642b12325e0 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:51.769416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 680/1356] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:43:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:51.776327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7963cd5-cd7b-4763-baa4-a71c02023204 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:51.777142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7963cd5-cd7b-4763-baa4-a71c02023204 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:51.777588 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7963cd5-cd7b-4763-baa4-a71c02023204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:51.827628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:51.866565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-e7963cd5-cd7b-4763-baa4-a71c02023204 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:51.867969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e7963cd5-cd7b-4763-baa4-a71c02023204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:51.970703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 677/1357] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:43:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:52.027193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb89fa3-c3d0-4b2f-ab7b-b683ecd09bc0 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:52.028510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb89fa3-c3d0-4b2f-ab7b-b683ecd09bc0 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:52.029677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb89fa3-c3d0-4b2f-ab7b-b683ecd09bc0 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:52.032072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 681/1358] 10.222.0.64 () {54 vars in 961 bytes} [Thu Jul 14 17:43:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:52.039720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a662c10-ba96-45ee-857d-e75fe1cc5bec None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:52.040058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a662c10-ba96-45ee-857d-e75fe1cc5bec None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:52.040345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a662c10-ba96-45ee-857d-e75fe1cc5bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:52.134954 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-1a662c10-ba96-45ee-857d-e75fe1cc5bec None None] MFA Rules not processed for user `6423f54152064b909d03c0908d636993`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:52.136199 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1a662c10-ba96-45ee-857d-e75fe1cc5bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:52.196157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:52.278864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 678/1359] 10.222.0.64 () {58 vars in 1058 bytes} [Thu Jul 14 17:43:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:52.291460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb89fa3-c3d0-4b2f-ab7b-b683ecd09bc0 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:43:52.294549 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9bb89fa3-c3d0-4b2f-ab7b-b683ecd09bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:52.451876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb89fa3-c3d0-4b2f-ab7b-b683ecd09bc0 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:43:52.535650 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:52.579473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31721eb-ccfc-4341-b14b-bc5ab8f88fe0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:43:52.580321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31721eb-ccfc-4341-b14b-bc5ab8f88fe0 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:52.580623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31721eb-ccfc-4341-b14b-bc5ab8f88fe0 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:52.580875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31721eb-ccfc-4341-b14b-bc5ab8f88fe0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:52.582500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f31721eb-ccfc-4341-b14b-bc5ab8f88fe0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:52.679180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31721eb-ccfc-4341-b14b-bc5ab8f88fe0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:43:52.680818 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31721eb-ccfc-4341-b14b-bc5ab8f88fe0 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:43:52.699896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 682/1360] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:43:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:43:55.004896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01b492-e658-4cb5-979f-572570952f60 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:55.005786 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01b492-e658-4cb5-979f-572570952f60 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:55.005786 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01b492-e658-4cb5-979f-572570952f60 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:55.006846 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 679/1361] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:43:55.012575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088687e-06d4-4b17-9f4b-05180b872b51 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:55.013291 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088687e-06d4-4b17-9f4b-05180b872b51 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:55.013291 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088687e-06d4-4b17-9f4b-05180b872b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:55.108342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-a088687e-06d4-4b17-9f4b-05180b872b51 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:55.110333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a088687e-06d4-4b17-9f4b-05180b872b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:55.160337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:55.264201 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 683/1362] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:43:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:43:55.344308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01b492-e658-4cb5-979f-572570952f60 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:43:55.347082 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6e01b492-e658-4cb5-979f-572570952f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:55.565269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01b492-e658-4cb5-979f-572570952f60 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:43:55.576288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:55.722250 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcca57a-d0f2-49c0-8e38-f3671c59fbee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:43:55.723030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcca57a-d0f2-49c0-8e38-f3671c59fbee service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:55.723315 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcca57a-d0f2-49c0-8e38-f3671c59fbee service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:55.723590 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcca57a-d0f2-49c0-8e38-f3671c59fbee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:55.725323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-2bcca57a-d0f2-49c0-8e38-f3671c59fbee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:55.841050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcca57a-d0f2-49c0-8e38-f3671c59fbee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:43:55.842825 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcca57a-d0f2-49c0-8e38-f3671c59fbee service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:43:55.870809 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 680/1363] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:43:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:43:58.304134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1061a-f8d1-4e42-9be2-a66eed6c7134 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:58.305163 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1061a-f8d1-4e42-9be2-a66eed6c7134 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:58.305451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1061a-f8d1-4e42-9be2-a66eed6c7134 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:58.307068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 684/1364] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:43:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:58.314219 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4da71-0cc6-4339-920e-866af4b1a80d None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:58.314764 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4da71-0cc6-4339-920e-866af4b1a80d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:58.315220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4da71-0cc6-4339-920e-866af4b1a80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:58.342416 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:58.416191 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-5ca4da71-0cc6-4339-920e-866af4b1a80d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:58.418111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5ca4da71-0cc6-4339-920e-866af4b1a80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:58.616024 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 681/1365] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:43:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:58.665387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f3c40-6aef-423e-9108-2d272bf5d28b None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:58.665688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f3c40-6aef-423e-9108-2d272bf5d28b None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:58.666033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f3c40-6aef-423e-9108-2d272bf5d28b None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:58.668318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 685/1366] 10.222.0.64 () {54 vars in 961 bytes} [Thu Jul 14 17:43:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:43:58.677664 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b82a33-8f2e-41ca-b116-c87c1a35ba7a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:58.678084 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b82a33-8f2e-41ca-b116-c87c1a35ba7a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:58.678261 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b82a33-8f2e-41ca-b116-c87c1a35ba7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:58.776830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-61b82a33-8f2e-41ca-b116-c87c1a35ba7a None None] MFA Rules not processed for user `6423f54152064b909d03c0908d636993`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:43:58.778085 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-61b82a33-8f2e-41ca-b116-c87c1a35ba7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:58.821990 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:58.931534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 682/1367] 10.222.0.64 () {58 vars in 1058 bytes} [Thu Jul 14 17:43:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:43:58.944817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f3c40-6aef-423e-9108-2d272bf5d28b None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:43:58.946524 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7f0f3c40-6aef-423e-9108-2d272bf5d28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:59.171303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f3c40-6aef-423e-9108-2d272bf5d28b None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:43:59.197624 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:43:59.317754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846c839-8e75-47f4-80be-5702f8ff12d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:43:59.318732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846c839-8e75-47f4-80be-5702f8ff12d1 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:43:59.319270 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846c839-8e75-47f4-80be-5702f8ff12d1 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:43:59.319673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846c839-8e75-47f4-80be-5702f8ff12d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:43:59.321626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3846c839-8e75-47f4-80be-5702f8ff12d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:43:59.432000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846c839-8e75-47f4-80be-5702f8ff12d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:43:59.433849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846c839-8e75-47f4-80be-5702f8ff12d1 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:43:59.454896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 686/1368] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:44:00.671313 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6c82f-3f6c-4404-85fb-f85d6d5176a3 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:00.672158 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6c82f-3f6c-4404-85fb-f85d6d5176a3 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:00.672158 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff6c82f-3f6c-4404-85fb-f85d6d5176a3 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:00.674061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 683/1369] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:44:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:44:00.679438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c0820-cf7a-4aa7-9dd4-4277ebb6f064 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:00.679962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c0820-cf7a-4aa7-9dd4-4277ebb6f064 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:00.680061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c0820-cf7a-4aa7-9dd4-4277ebb6f064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:00.718134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:00.764990 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-db0c0820-cf7a-4aa7-9dd4-4277ebb6f064 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:00.766868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-db0c0820-cf7a-4aa7-9dd4-4277ebb6f064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:00.907965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 687/1370] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:01.277435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad622bc-ca97-42a4-8d4d-d10d81839a00 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:01.278336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad622bc-ca97-42a4-8d4d-d10d81839a00 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:01.279012 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad622bc-ca97-42a4-8d4d-d10d81839a00 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:01.281041 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 684/1371] 10.222.0.64 () {54 vars in 963 bytes} [Thu Jul 14 17:44:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:01.292743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb89e15-0416-4aa0-964e-db772f996e20 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:01.293029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb89e15-0416-4aa0-964e-db772f996e20 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:01.293327 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb89e15-0416-4aa0-964e-db772f996e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:01.415742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-bfb89e15-0416-4aa0-964e-db772f996e20 None None] MFA Rules not processed for user `e49fad7db3c643b99e9071d9ca5f0196`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:01.421411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-bfb89e15-0416-4aa0-964e-db772f996e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:01.480717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:01.526871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 688/1372] 10.222.0.64 () {58 vars in 1060 bytes} [Thu Jul 14 17:44:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:44:01.536234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad622bc-ca97-42a4-8d4d-d10d81839a00 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:01.537768 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1ad622bc-ca97-42a4-8d4d-d10d81839a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:01.666106 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad622bc-ca97-42a4-8d4d-d10d81839a00 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:01.769712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:01.812430 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503f0e1-d2a9-4738-adc0-4c720818de02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:01.813814 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503f0e1-d2a9-4738-adc0-4c720818de02 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:01.814038 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503f0e1-d2a9-4738-adc0-4c720818de02 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:01.814551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503f0e1-d2a9-4738-adc0-4c720818de02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:01.820350 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0503f0e1-d2a9-4738-adc0-4c720818de02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:01.854896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53630d7-dd1f-4398-9e34-2f9033c2a007 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:01.855432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53630d7-dd1f-4398-9e34-2f9033c2a007 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:01.855661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53630d7-dd1f-4398-9e34-2f9033c2a007 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:01.858135 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 689/1373] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:44:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:01.865236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c346896-4f9d-46fc-819f-50ad5958acf3 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:01.865833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c346896-4f9d-46fc-819f-50ad5958acf3 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:01.866161 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c346896-4f9d-46fc-819f-50ad5958acf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:01.974735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503f0e1-d2a9-4738-adc0-4c720818de02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:01.976499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0503f0e1-d2a9-4738-adc0-4c720818de02 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:01.991714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4c346896-4f9d-46fc-819f-50ad5958acf3 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:01.993663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4c346896-4f9d-46fc-819f-50ad5958acf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:02.000240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 685/1374] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:02.127870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 690/1375] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:44:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:44:02.176559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503f0e1-d2a9-4738-adc0-4c720818de02 service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:02.177956 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0503f0e1-d2a9-4738-adc0-4c720818de02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:02.286144 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:02.330613 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503f0e1-d2a9-4738-adc0-4c720818de02 service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:02.479135 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6a4bc3-0d09-4045-886e-3ccfa76844c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:02.479945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a4bc3-0d09-4045-886e-3ccfa76844c1 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:02.480304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a4bc3-0d09-4045-886e-3ccfa76844c1 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:02.480734 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6a4bc3-0d09-4045-886e-3ccfa76844c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:02.482982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1c6a4bc3-0d09-4045-886e-3ccfa76844c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:02.600526 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:02.614311 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a4bc3-0d09-4045-886e-3ccfa76844c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:02.616055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6a4bc3-0d09-4045-886e-3ccfa76844c1 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:02.644681 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 686/1376] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:04.972611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fca7a7-8be4-4412-9989-7cd44cce0bd8 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:04.973477 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fca7a7-8be4-4412-9989-7cd44cce0bd8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:04.973942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fca7a7-8be4-4412-9989-7cd44cce0bd8 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:04.975876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 691/1377] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:44:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:04.981881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069be8f4-808e-41d1-804f-3a5b8712f9b8 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:04.982718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069be8f4-808e-41d1-804f-3a5b8712f9b8 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:04.983146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069be8f4-808e-41d1-804f-3a5b8712f9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:05.068698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-069be8f4-808e-41d1-804f-3a5b8712f9b8 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:05.071108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-069be8f4-808e-41d1-804f-3a5b8712f9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:05.203865 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 687/1378] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:05.256442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fca7a7-8be4-4412-9989-7cd44cce0bd8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:05.258145 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-29fca7a7-8be4-4412-9989-7cd44cce0bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:05.340352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:05.483436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fca7a7-8be4-4412-9989-7cd44cce0bd8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:05.668446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3f1759-cca7-4f1e-8af3-a2da53b42fab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:05.669268 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f1759-cca7-4f1e-8af3-a2da53b42fab service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:05.669556 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f1759-cca7-4f1e-8af3-a2da53b42fab service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:05.669882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3f1759-cca7-4f1e-8af3-a2da53b42fab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:05.673057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6f3f1759-cca7-4f1e-8af3-a2da53b42fab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:05.692608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:05.814913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f1759-cca7-4f1e-8af3-a2da53b42fab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:05.816603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3f1759-cca7-4f1e-8af3-a2da53b42fab service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:05.846672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 692/1379] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:08.161327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc9c37-ae03-4293-a9fd-e1e30e04890d None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:08.161874 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc9c37-ae03-4293-a9fd-e1e30e04890d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:08.161874 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc9c37-ae03-4293-a9fd-e1e30e04890d None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:08.163559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 688/1380] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:08.169266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab260d-5517-4062-87d1-ed99dafcc8df None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:08.169841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab260d-5517-4062-87d1-ed99dafcc8df None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:08.169841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab260d-5517-4062-87d1-ed99dafcc8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:08.269064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-55ab260d-5517-4062-87d1-ed99dafcc8df None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:08.270647 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-55ab260d-5517-4062-87d1-ed99dafcc8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:08.389317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:08.407987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 693/1381] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:44:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:08.455340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc9c37-ae03-4293-a9fd-e1e30e04890d None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:08.456992 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-91fc9c37-ae03-4293-a9fd-e1e30e04890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:08.552822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:08.665834 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc9c37-ae03-4293-a9fd-e1e30e04890d None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:08.850230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2408394c-ce22-48dc-82a2-3e15d299ddb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:08.851034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408394c-ce22-48dc-82a2-3e15d299ddb0 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:08.851340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408394c-ce22-48dc-82a2-3e15d299ddb0 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:08.851704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2408394c-ce22-48dc-82a2-3e15d299ddb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:08.853599 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-2408394c-ce22-48dc-82a2-3e15d299ddb0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:08.884682 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:08.976225 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408394c-ce22-48dc-82a2-3e15d299ddb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:08.977862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2408394c-ce22-48dc-82a2-3e15d299ddb0 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:09.003155 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 689/1382] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:11.360935 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f6a71-8e31-45ec-93bb-24dfde71d461 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:11.361482 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f6a71-8e31-45ec-93bb-24dfde71d461 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:11.361599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f6a71-8e31-45ec-93bb-24dfde71d461 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:11.363160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 694/1383] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:44:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:11.369575 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc351feb-5ca9-4814-ac2b-262278070586 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:11.369980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc351feb-5ca9-4814-ac2b-262278070586 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:11.370140 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc351feb-5ca9-4814-ac2b-262278070586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:11.440640 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-cc351feb-5ca9-4814-ac2b-262278070586 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:11.441757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-cc351feb-5ca9-4814-ac2b-262278070586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:11.540263 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:11.544407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 690/1384] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:11.588511 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f6a71-8e31-45ec-93bb-24dfde71d461 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:11.590174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-2a0f6a71-8e31-45ec-93bb-24dfde71d461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:11.745816 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f6a71-8e31-45ec-93bb-24dfde71d461 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:11.820987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:11.883089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9783841e-5c4b-4ecb-8447-969032c8244e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:11.883893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783841e-5c4b-4ecb-8447-969032c8244e service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:11.884269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783841e-5c4b-4ecb-8447-969032c8244e service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:11.884606 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9783841e-5c4b-4ecb-8447-969032c8244e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:11.886416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9783841e-5c4b-4ecb-8447-969032c8244e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:11.985494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783841e-5c4b-4ecb-8447-969032c8244e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:11.987336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9783841e-5c4b-4ecb-8447-969032c8244e service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:12.006759 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 695/1385] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:14.073108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9ed5c-d76f-4300-80df-8baa7505a424 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:14.073680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9ed5c-d76f-4300-80df-8baa7505a424 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:14.073680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff9ed5c-d76f-4300-80df-8baa7505a424 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:14.075063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 691/1386] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:14.080618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be12f83-123e-4062-a432-973b241fd0b2 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:14.081211 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be12f83-123e-4062-a432-973b241fd0b2 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:14.081211 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be12f83-123e-4062-a432-973b241fd0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:14.146814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4be12f83-123e-4062-a432-973b241fd0b2 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:14.148296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4be12f83-123e-4062-a432-973b241fd0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:14.174843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:14.248640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 696/1387] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:44:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:14.294093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9ed5c-d76f-4300-80df-8baa7505a424 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:14.296209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5ff9ed5c-d76f-4300-80df-8baa7505a424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:14.473156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff9ed5c-d76f-4300-80df-8baa7505a424 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:14.581267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:14.632757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f0a16-273f-4ba2-9cda-170ee6cec2a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:14.633341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f0a16-273f-4ba2-9cda-170ee6cec2a5 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:14.633554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f0a16-273f-4ba2-9cda-170ee6cec2a5 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:14.633830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f0a16-273f-4ba2-9cda-170ee6cec2a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:14.635484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a22f0a16-273f-4ba2-9cda-170ee6cec2a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:14.744152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f0a16-273f-4ba2-9cda-170ee6cec2a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:14.746265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f0a16-273f-4ba2-9cda-170ee6cec2a5 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:14.773977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 692/1388] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:16.996090 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b63ada-8c70-42d3-b814-b1cf286bc796 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:16.996634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b63ada-8c70-42d3-b814-b1cf286bc796 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:16.996634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b63ada-8c70-42d3-b814-b1cf286bc796 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:16.998030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 697/1389] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:44:17.004214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c64386-df48-42fb-bb43-68769d1f58c1 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:17.004682 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c64386-df48-42fb-bb43-68769d1f58c1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:17.004785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c64386-df48-42fb-bb43-68769d1f58c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:17.078172 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-95c64386-df48-42fb-bb43-68769d1f58c1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:17.080183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-95c64386-df48-42fb-bb43-68769d1f58c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:17.121562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:17.193419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 693/1390] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:17.240872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b63ada-8c70-42d3-b814-b1cf286bc796 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:17.242520 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e3b63ada-8c70-42d3-b814-b1cf286bc796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:17.403495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b63ada-8c70-42d3-b814-b1cf286bc796 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:17.413503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:17.525844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556185ea-4269-4897-96f6-b62c36a45535 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:17.526927 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556185ea-4269-4897-96f6-b62c36a45535 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:17.526927 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556185ea-4269-4897-96f6-b62c36a45535 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:17.527215 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556185ea-4269-4897-96f6-b62c36a45535 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:17.529441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-556185ea-4269-4897-96f6-b62c36a45535 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:17.624918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556185ea-4269-4897-96f6-b62c36a45535 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:17.626602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556185ea-4269-4897-96f6-b62c36a45535 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:17.647369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 698/1391] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:17.837054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62702191-b300-4c14-af81-19373b02ebbd None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:17.837372 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62702191-b300-4c14-af81-19373b02ebbd None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:17.837572 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62702191-b300-4c14-af81-19373b02ebbd None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:17.839186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 694/1392] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:44:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:17.846187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95fb70-551c-49b6-9e59-c1717a19b105 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:17.846656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95fb70-551c-49b6-9e59-c1717a19b105 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:17.847056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d95fb70-551c-49b6-9e59-c1717a19b105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:17.866640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:17.945582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-1d95fb70-551c-49b6-9e59-c1717a19b105 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:17.946826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1d95fb70-551c-49b6-9e59-c1717a19b105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:18.054025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 699/1393] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:18.118395 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115b1be-32b1-47b3-820a-5ae51eb9a5c3 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:18.119101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115b1be-32b1-47b3-820a-5ae51eb9a5c3 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:18.119101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115b1be-32b1-47b3-820a-5ae51eb9a5c3 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:18.120693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 695/1394] 10.222.0.64 () {54 vars in 962 bytes} [Thu Jul 14 17:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:44:18.127922 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0987d0c-fc78-4e83-bf3c-3d39b2789801 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:18.128173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0987d0c-fc78-4e83-bf3c-3d39b2789801 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:18.128454 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0987d0c-fc78-4e83-bf3c-3d39b2789801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:18.221536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-c0987d0c-fc78-4e83-bf3c-3d39b2789801 None None] MFA Rules not processed for user `b2b4bf2aa1744a46b6d2a1b5a136a0e1`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:18.222669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c0987d0c-fc78-4e83-bf3c-3d39b2789801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:18.259657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:18.318679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 700/1395] 10.222.0.64 () {58 vars in 1059 bytes} [Thu Jul 14 17:44:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:18.327018 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115b1be-32b1-47b3-820a-5ae51eb9a5c3 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:18.329761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0115b1be-32b1-47b3-820a-5ae51eb9a5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:18.458130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115b1be-32b1-47b3-820a-5ae51eb9a5c3 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:18.537364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:18.551714 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc939aaa-3cd0-4f73-93ba-d058bda1c520 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2b4bf2aa1744a46b6d2a1b5a136a0e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:18.552398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939aaa-3cd0-4f73-93ba-d058bda1c520 service glance] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:18.552679 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939aaa-3cd0-4f73-93ba-d058bda1c520 service glance] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:18.552953 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc939aaa-3cd0-4f73-93ba-d058bda1c520 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:18.554634 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-dc939aaa-3cd0-4f73-93ba-d058bda1c520 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:18.675630 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc939aaa-3cd0-4f73-93ba-d058bda1c520 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:18.677638 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc939aaa-3cd0-4f73-93ba-d058bda1c520 service glance] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:18.698643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 696/1396] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:44:19.433848 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8baae01-f17d-4a89-881a-124d384ace7e None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:19.434384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8baae01-f17d-4a89-881a-124d384ace7e None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:19.434384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8baae01-f17d-4a89-881a-124d384ace7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:19.506813 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-f8baae01-f17d-4a89-881a-124d384ace7e None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:19.508101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f8baae01-f17d-4a89-881a-124d384ace7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:19.604460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 701/1397] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 17:44:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:19.651692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da095a5-816c-4e0e-a8aa-a48a6fb2edc8 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:19.652248 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da095a5-816c-4e0e-a8aa-a48a6fb2edc8 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:19.652358 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da095a5-816c-4e0e-a8aa-a48a6fb2edc8 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:19.653972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 697/1398] 10.222.0.64 () {54 vars in 942 bytes} [Thu Jul 14 17:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:19.661125 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec0e80-26e0-48db-b9b2-1e05fe7934b5 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:19.661699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec0e80-26e0-48db-b9b2-1e05fe7934b5 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:19.661994 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec0e80-26e0-48db-b9b2-1e05fe7934b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:19.726830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:19.748629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-ddec0e80-26e0-48db-b9b2-1e05fe7934b5 None None] MFA Rules not processed for user `97f15306528e415baa5c69f12f22c75e`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:19.749865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ddec0e80-26e0-48db-b9b2-1e05fe7934b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:19.816471 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de0458-a6f5-4796-9d93-d1541fb570e5 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:19.816754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de0458-a6f5-4796-9d93-d1541fb570e5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:19.817087 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de0458-a6f5-4796-9d93-d1541fb570e5 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:19.818502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 698/1399] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:44:19.824949 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9e1da-b380-42d5-aaa2-160d1075ef6d None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:19.825242 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9e1da-b380-42d5-aaa2-160d1075ef6d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:19.825519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9e1da-b380-42d5-aaa2-160d1075ef6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:19.909008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-a8e9e1da-b380-42d5-aaa2-160d1075ef6d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:19.910185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a8e9e1da-b380-42d5-aaa2-160d1075ef6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:19.912566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 702/1400] 10.222.0.64 () {58 vars in 1039 bytes} [Thu Jul 14 17:44:19 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:19.920773 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec0e80-26e0-48db-b9b2-1e05fe7934b5 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:19.935324 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec0e80-26e0-48db-b9b2-1e05fe7934b5 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:19.957516 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:20.019985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 699/1401] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:44:20.047467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:20.055295 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2ff93-0979-4446-9c1a-dbbb2ea31b5e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:20.056390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2ff93-0979-4446-9c1a-dbbb2ea31b5e service swift] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:20.057001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2ff93-0979-4446-9c1a-dbbb2ea31b5e service swift] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:20.057292 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2ff93-0979-4446-9c1a-dbbb2ea31b5e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:20.059254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f8b2ff93-0979-4446-9c1a-dbbb2ea31b5e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:20.069622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9e1da-b380-42d5-aaa2-160d1075ef6d None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:20.070833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a8e9e1da-b380-42d5-aaa2-160d1075ef6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:20.159238 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2ff93-0979-4446-9c1a-dbbb2ea31b5e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:20.160817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2ff93-0979-4446-9c1a-dbbb2ea31b5e service swift] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:20.169458 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 703/1402] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 17:44:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 17:44:20.209903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9e1da-b380-42d5-aaa2-160d1075ef6d None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:20.283795 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:20.364762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9620e6-127c-4c78-b4b3-0e821df277c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:20.365849 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9620e6-127c-4c78-b4b3-0e821df277c3 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:20.366196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9620e6-127c-4c78-b4b3-0e821df277c3 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:20.366592 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9620e6-127c-4c78-b4b3-0e821df277c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:20.368569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3f9620e6-127c-4c78-b4b3-0e821df277c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:20.475335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9620e6-127c-4c78-b4b3-0e821df277c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:20.477049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9620e6-127c-4c78-b4b3-0e821df277c3 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:20.497705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 700/1403] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:44:22.649314 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c52e7-2880-4d27-bffd-cb0a778881c1 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:22.649842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c52e7-2880-4d27-bffd-cb0a778881c1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:22.649842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78c52e7-2880-4d27-bffd-cb0a778881c1 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:22.651412 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 704/1404] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:22.656878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac6513-90ee-49dc-9d51-14e787fdd545 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:22.657589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac6513-90ee-49dc-9d51-14e787fdd545 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:22.657589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac6513-90ee-49dc-9d51-14e787fdd545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:22.736782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-dbac6513-90ee-49dc-9d51-14e787fdd545 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:22.738033 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-dbac6513-90ee-49dc-9d51-14e787fdd545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:22.760344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:22.936247 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 701/1405] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:44:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:44:22.984430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c52e7-2880-4d27-bffd-cb0a778881c1 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:22.988084 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f78c52e7-2880-4d27-bffd-cb0a778881c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:23.182353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:23.197404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78c52e7-2880-4d27-bffd-cb0a778881c1 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:23.396685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca390a58-c765-4de1-a090-bfb46940f1ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:23.397483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca390a58-c765-4de1-a090-bfb46940f1ec service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:23.397832 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca390a58-c765-4de1-a090-bfb46940f1ec service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:23.398378 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca390a58-c765-4de1-a090-bfb46940f1ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:23.400251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ca390a58-c765-4de1-a090-bfb46940f1ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:23.518430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca390a58-c765-4de1-a090-bfb46940f1ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:23.520430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca390a58-c765-4de1-a090-bfb46940f1ec service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:23.533304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:23.544409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 705/1406] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:26.178409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8c801-40e6-45e7-9b80-6d3960303f7d None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:26.178976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8c801-40e6-45e7-9b80-6d3960303f7d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:26.178976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b8c801-40e6-45e7-9b80-6d3960303f7d None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:26.180798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 702/1407] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:26.187859 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e26b4e-544c-4839-a020-59e5642fd455 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:26.188815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e26b4e-544c-4839-a020-59e5642fd455 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:26.189359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e26b4e-544c-4839-a020-59e5642fd455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:26.302474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-57e26b4e-544c-4839-a020-59e5642fd455 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:26.303699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-57e26b4e-544c-4839-a020-59e5642fd455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:26.461075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 706/1408] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:44:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:26.531876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8c801-40e6-45e7-9b80-6d3960303f7d None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:26.534062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-95b8c801-40e6-45e7-9b80-6d3960303f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:26.715045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:26.716563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b8c801-40e6-45e7-9b80-6d3960303f7d None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:26.856733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55953236-e022-4838-a21c-71675a40c590 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:26.857555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55953236-e022-4838-a21c-71675a40c590 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:26.857845 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55953236-e022-4838-a21c-71675a40c590 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:26.858120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55953236-e022-4838-a21c-71675a40c590 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:26.859818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-55953236-e022-4838-a21c-71675a40c590 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:26.987260 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55953236-e022-4838-a21c-71675a40c590 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:26.990770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55953236-e022-4838-a21c-71675a40c590 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:27.028145 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 703/1409] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:36.459249 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=67 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:37.014731 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=98 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:43.071659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e90df7-5ca8-4547-8bef-6bb51bb12935 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:43.072310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e90df7-5ca8-4547-8bef-6bb51bb12935 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:43.072310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e90df7-5ca8-4547-8bef-6bb51bb12935 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:43.073630 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 707/1410] 10.222.0.64 () {54 vars in 918 bytes} [Thu Jul 14 17:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:43.079045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b37ce-5342-4fda-9122-2bcd83402405 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:43.079479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b37ce-5342-4fda-9122-2bcd83402405 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:43.079591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b37ce-5342-4fda-9122-2bcd83402405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:43.155762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-8c3b37ce-5342-4fda-9122-2bcd83402405 None None] MFA Rules not processed for user `619e3bec841343c5ad144f1d4e4d436d`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:43.157022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8c3b37ce-5342-4fda-9122-2bcd83402405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:43.261505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 704/1411] 10.222.0.64 () {58 vars in 1015 bytes} [Thu Jul 14 17:44:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:43.277042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec291dd-c093-424c-b993-17733370bba5 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:43.277337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec291dd-c093-424c-b993-17733370bba5 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:43.277646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec291dd-c093-424c-b993-17733370bba5 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:43.279027 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 708/1412] 10.222.0.64 () {54 vars in 918 bytes} [Thu Jul 14 17:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:43.283889 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf713a-2a45-43d0-b2d0-622fc07024c9 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:43.284187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf713a-2a45-43d0-b2d0-622fc07024c9 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:43.284535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf713a-2a45-43d0-b2d0-622fc07024c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:43.359374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:43.363413 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-8bdf713a-2a45-43d0-b2d0-622fc07024c9 None None] MFA Rules not processed for user `6423f54152064b909d03c0908d636993`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:43.365077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8bdf713a-2a45-43d0-b2d0-622fc07024c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:43.496890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 705/1413] 10.222.0.64 () {58 vars in 1015 bytes} [Thu Jul 14 17:44:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:46.043680 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b996a-ecdd-4a2c-a602-087276ba3f85 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:46.044423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b996a-ecdd-4a2c-a602-087276ba3f85 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:46.044423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b996a-ecdd-4a2c-a602-087276ba3f85 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:46.045632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:46.051043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8bbde-8277-4ba8-898e-ad68b4729851 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:46.051505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8bbde-8277-4ba8-898e-ad68b4729851 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:46.051604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f8bbde-8277-4ba8-898e-ad68b4729851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:46.121756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-39f8bbde-8277-4ba8-898e-ad68b4729851 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:46.123038 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-39f8bbde-8277-4ba8-898e-ad68b4729851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:46.181466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:46.225745 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:44:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:46.270303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b996a-ecdd-4a2c-a602-087276ba3f85 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:46.271837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-765b996a-ecdd-4a2c-a602-087276ba3f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:46.423060 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765b996a-ecdd-4a2c-a602-087276ba3f85 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:46.505106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:46.546493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543208d-8517-408b-8442-b20518f9eb07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:46.547152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543208d-8517-408b-8442-b20518f9eb07 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:46.547441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543208d-8517-408b-8442-b20518f9eb07 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:46.547723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543208d-8517-408b-8442-b20518f9eb07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:46.549444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d543208d-8517-408b-8442-b20518f9eb07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:46.642910 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543208d-8517-408b-8442-b20518f9eb07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:46.644721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543208d-8517-408b-8442-b20518f9eb07 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:46.663492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 710/1416] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:47.577375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f9581-29d8-4da4-b3da-2de445025500 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:47.577924 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f9581-29d8-4da4-b3da-2de445025500 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:47.577924 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f9581-29d8-4da4-b3da-2de445025500 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:47.579351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 17:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:47.584948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e57bc-0791-46ea-8fda-ac7a1e018bf5 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:47.585493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e57bc-0791-46ea-8fda-ac7a1e018bf5 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:47.585493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e57bc-0791-46ea-8fda-ac7a1e018bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:47.663079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-c58e57bc-0791-46ea-8fda-ac7a1e018bf5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:47.664363 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c58e57bc-0791-46ea-8fda-ac7a1e018bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:47.695919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:47.769914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 17:44:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:44:47.921310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f9581-29d8-4da4-b3da-2de445025500 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:47.922747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3c6f9581-29d8-4da4-b3da-2de445025500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:48.083634 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f9581-29d8-4da4-b3da-2de445025500 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:48.125262 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:48.199595 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c346364-fa01-4ed9-99b6-a22d32771946 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:48.200270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c346364-fa01-4ed9-99b6-a22d32771946 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:48.200562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c346364-fa01-4ed9-99b6-a22d32771946 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:48.200826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c346364-fa01-4ed9-99b6-a22d32771946 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:48.202475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6c346364-fa01-4ed9-99b6-a22d32771946 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:48.290335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c346364-fa01-4ed9-99b6-a22d32771946 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:48.291961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c346364-fa01-4ed9-99b6-a22d32771946 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:48.309871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 708/1419] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:49.225866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bc0cb-5b0e-4acb-9e52-a8235865fa68 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:49.226400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bc0cb-5b0e-4acb-9e52-a8235865fa68 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:49.226513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bc0cb-5b0e-4acb-9e52-a8235865fa68 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:49.227821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 17:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:49.232832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f323f1f-215f-4369-9763-f9c8ef8040f1 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:49.233244 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f323f1f-215f-4369-9763-f9c8ef8040f1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:49.233397 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f323f1f-215f-4369-9763-f9c8ef8040f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:49.286135 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:49.308118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-5f323f1f-215f-4369-9763-f9c8ef8040f1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:49.309334 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5f323f1f-215f-4369-9763-f9c8ef8040f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:49.403985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 17:44:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:49.556587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bc0cb-5b0e-4acb-9e52-a8235865fa68 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:49.558138 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-df9bc0cb-5b0e-4acb-9e52-a8235865fa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:49.718153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bc0cb-5b0e-4acb-9e52-a8235865fa68 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:49.731302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:49.834510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7ef78-0cc7-4fd3-8cca-6f1f739d5008 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:49.835168 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7ef78-0cc7-4fd3-8cca-6f1f739d5008 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:49.835449 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7ef78-0cc7-4fd3-8cca-6f1f739d5008 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:49.835722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7ef78-0cc7-4fd3-8cca-6f1f739d5008 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:49.837423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-dfd7ef78-0cc7-4fd3-8cca-6f1f739d5008 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:49.929215 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7ef78-0cc7-4fd3-8cca-6f1f739d5008 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:49.930745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7ef78-0cc7-4fd3-8cca-6f1f739d5008 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:49.949110 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 713/1422] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:50.826553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c238be5-dede-483d-a10a-0eff42cd96dc None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:50.827215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c238be5-dede-483d-a10a-0eff42cd96dc None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:50.827215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c238be5-dede-483d-a10a-0eff42cd96dc None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:50.828545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 17:44:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:50.834953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd143d3a-d98e-4ff3-ae7c-39422b42de9f None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:50.835428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd143d3a-d98e-4ff3-ae7c-39422b42de9f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:50.835770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd143d3a-d98e-4ff3-ae7c-39422b42de9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:50.855044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:50.917948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-dd143d3a-d98e-4ff3-ae7c-39422b42de9f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:50.919180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-dd143d3a-d98e-4ff3-ae7c-39422b42de9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:51.017796 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 17:44:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:51.168811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c238be5-dede-483d-a10a-0eff42cd96dc None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:51.170323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-2c238be5-dede-483d-a10a-0eff42cd96dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:51.280030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:51.315865 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c238be5-dede-483d-a10a-0eff42cd96dc None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:51.460246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c418463-6394-47e0-a3b6-e0654da0a05b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:51.460906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c418463-6394-47e0-a3b6-e0654da0a05b service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:51.461186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c418463-6394-47e0-a3b6-e0654da0a05b service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:51.461462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c418463-6394-47e0-a3b6-e0654da0a05b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:51.463122 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6c418463-6394-47e0-a3b6-e0654da0a05b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:51.541032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:51.555121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c418463-6394-47e0-a3b6-e0654da0a05b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:51.556775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c418463-6394-47e0-a3b6-e0654da0a05b service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:51.575087 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 711/1425] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:52.458123 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af50900-414f-429f-8bbf-881f00e4e9b7 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:52.458675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af50900-414f-429f-8bbf-881f00e4e9b7 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:52.458675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af50900-414f-429f-8bbf-881f00e4e9b7 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:52.460200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 17:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:52.466037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a675cd4-a358-4d75-ab25-229a67db9064 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:52.466752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a675cd4-a358-4d75-ab25-229a67db9064 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:52.466895 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a675cd4-a358-4d75-ab25-229a67db9064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:52.549921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-5a675cd4-a358-4d75-ab25-229a67db9064 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:52.551174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5a675cd4-a358-4d75-ab25-229a67db9064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:52.646687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 17:44:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:44:52.800885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af50900-414f-429f-8bbf-881f00e4e9b7 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:52.802427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5af50900-414f-429f-8bbf-881f00e4e9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:52.890705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:52.964256 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af50900-414f-429f-8bbf-881f00e4e9b7 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:53.076828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c009ed-edf7-47be-86a9-6f800ca13886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:53.077472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c009ed-edf7-47be-86a9-6f800ca13886 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:53.077754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c009ed-edf7-47be-86a9-6f800ca13886 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:53.078027 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c009ed-edf7-47be-86a9-6f800ca13886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:53.079669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-07c009ed-edf7-47be-86a9-6f800ca13886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:53.135108 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:53.177689 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c009ed-edf7-47be-86a9-6f800ca13886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:53.179300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c009ed-edf7-47be-86a9-6f800ca13886 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:53.197898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 716/1428] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:54.032999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4269fc2-10d5-4ff1-8b5e-962c51aec8a9 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:54.033959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4269fc2-10d5-4ff1-8b5e-962c51aec8a9 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:54.033959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4269fc2-10d5-4ff1-8b5e-962c51aec8a9 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:54.036216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 17:44:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:44:54.042919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72967f-6703-4186-ad51-5356da7ba802 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:54.043517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72967f-6703-4186-ad51-5356da7ba802 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:54.043862 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72967f-6703-4186-ad51-5356da7ba802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:54.154440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-7a72967f-6703-4186-ad51-5356da7ba802 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:54.155687 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7a72967f-6703-4186-ad51-5356da7ba802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:54.255704 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 17:44:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:54.410303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4269fc2-10d5-4ff1-8b5e-962c51aec8a9 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:54.411781 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b4269fc2-10d5-4ff1-8b5e-962c51aec8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:54.463556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:54.570014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4269fc2-10d5-4ff1-8b5e-962c51aec8a9 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:54.698097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2cd11-37de-468e-a33c-e27b59765ec1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:54.698765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2cd11-37de-468e-a33c-e27b59765ec1 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:54.699050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2cd11-37de-468e-a33c-e27b59765ec1 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:54.699328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2cd11-37de-468e-a33c-e27b59765ec1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:54.701074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-74a2cd11-37de-468e-a33c-e27b59765ec1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:54.709271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:54.798980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2cd11-37de-468e-a33c-e27b59765ec1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:54.801220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2cd11-37de-468e-a33c-e27b59765ec1 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:54.821884 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 714/1431] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:44:55.662752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdace1-9900-4a1c-be5f-e0e6732e7905 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:55.663590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdace1-9900-4a1c-be5f-e0e6732e7905 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:55.663590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cdace1-9900-4a1c-be5f-e0e6732e7905 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:55.665679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 17:44:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:55.671373 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec33278-1239-4088-9786-29b3163dc827 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:55.671987 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec33278-1239-4088-9786-29b3163dc827 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:55.672142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec33278-1239-4088-9786-29b3163dc827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:55.756639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-2ec33278-1239-4088-9786-29b3163dc827 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:55.757885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-2ec33278-1239-4088-9786-29b3163dc827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:55.838467 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:55.874454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 17:44:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:56.024720 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cdace1-9900-4a1c-be5f-e0e6732e7905 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:56.026604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-36cdace1-9900-4a1c-be5f-e0e6732e7905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:56.045086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:56.199517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cdace1-9900-4a1c-be5f-e0e6732e7905 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:56.296591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:56.322923 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286797fc-9dd0-4b81-8e74-6280236dd6a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:56.323572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286797fc-9dd0-4b81-8e74-6280236dd6a6 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:56.323859 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286797fc-9dd0-4b81-8e74-6280236dd6a6 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:56.324169 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286797fc-9dd0-4b81-8e74-6280236dd6a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:56.325843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-286797fc-9dd0-4b81-8e74-6280236dd6a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:56.423073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286797fc-9dd0-4b81-8e74-6280236dd6a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:56.424725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286797fc-9dd0-4b81-8e74-6280236dd6a6 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:56.444668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 719/1434] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:44:57.284011 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bf08a-0f2b-4b84-aef9-18dfa1089b65 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:57.284613 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bf08a-0f2b-4b84-aef9-18dfa1089b65 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:57.284613 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bf08a-0f2b-4b84-aef9-18dfa1089b65 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:57.286574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 17:44:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:44:57.292540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2709-2cd5-46ff-8427-b5249f42f295 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:57.293441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2709-2cd5-46ff-8427-b5249f42f295 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:57.293441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe2709-2cd5-46ff-8427-b5249f42f295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:57.403730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-acbe2709-2cd5-46ff-8427-b5249f42f295 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:57.405800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-acbe2709-2cd5-46ff-8427-b5249f42f295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:57.468546 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:57.528317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 17:44:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:44:57.688295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bf08a-0f2b-4b84-aef9-18dfa1089b65 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:57.689879 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6a8bf08a-0f2b-4b84-aef9-18dfa1089b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:57.855414 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bf08a-0f2b-4b84-aef9-18dfa1089b65 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:57.914696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:57.977225 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5cc3a-78d6-44b5-be32-b4ef0086be63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:57.977894 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5cc3a-78d6-44b5-be32-b4ef0086be63 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:57.978183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5cc3a-78d6-44b5-be32-b4ef0086be63 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:57.978465 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5cc3a-78d6-44b5-be32-b4ef0086be63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:57.980268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9ec5cc3a-78d6-44b5-be32-b4ef0086be63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:58.078646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5cc3a-78d6-44b5-be32-b4ef0086be63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:58.080312 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5cc3a-78d6-44b5-be32-b4ef0086be63 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:58.100939 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 717/1437] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:44:58.943307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65bdc6-e17a-463d-82f7-d7a0d9b92ea0 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:58.944270 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65bdc6-e17a-463d-82f7-d7a0d9b92ea0 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:58.944432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a65bdc6-e17a-463d-82f7-d7a0d9b92ea0 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:58.945879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 17:44:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:44:58.951483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6bbba-d107-477d-b41f-bfa962d00845 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:58.952297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6bbba-d107-477d-b41f-bfa962d00845 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:58.952481 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be6bbba-d107-477d-b41f-bfa962d00845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:59.036380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-0be6bbba-d107-477d-b41f-bfa962d00845 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:44:59.037618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0be6bbba-d107-477d-b41f-bfa962d00845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:59.039770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:59.147531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 17:44:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:44:59.302269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65bdc6-e17a-463d-82f7-d7a0d9b92ea0 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:44:59.303896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6a65bdc6-e17a-463d-82f7-d7a0d9b92ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:59.561463 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a65bdc6-e17a-463d-82f7-d7a0d9b92ea0 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:44:59.609968 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:44:59.724421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251db61a-2885-46e1-a2e6-67327f3190a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:44:59.725101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251db61a-2885-46e1-a2e6-67327f3190a4 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:44:59.725392 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251db61a-2885-46e1-a2e6-67327f3190a4 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:44:59.725673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251db61a-2885-46e1-a2e6-67327f3190a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:44:59.727378 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-251db61a-2885-46e1-a2e6-67327f3190a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:44:59.876737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251db61a-2885-46e1-a2e6-67327f3190a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:44:59.878373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251db61a-2885-46e1-a2e6-67327f3190a4 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:44:59.898465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 722/1440] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:44:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:45:02.387047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e534190-2f40-4ba0-b1ad-8ede1fb4e777 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:02.387909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e534190-2f40-4ba0-b1ad-8ede1fb4e777 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:02.387909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e534190-2f40-4ba0-b1ad-8ede1fb4e777 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:02.388910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:45:02.395776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840826d-ee42-46b2-870d-0b4aa4b129ed None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:02.396368 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840826d-ee42-46b2-870d-0b4aa4b129ed None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:02.396797 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840826d-ee42-46b2-870d-0b4aa4b129ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:02.441837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:02.470392 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-9840826d-ee42-46b2-870d-0b4aa4b129ed None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:02.471634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9840826d-ee42-46b2-870d-0b4aa4b129ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:02.575674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:45:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:45:02.614757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e534190-2f40-4ba0-b1ad-8ede1fb4e777 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:02.616354 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7e534190-2f40-4ba0-b1ad-8ede1fb4e777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:02.722090 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e534190-2f40-4ba0-b1ad-8ede1fb4e777 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:02.754510 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:02.797948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb229502-67d5-4954-9b5e-a9251c84f10d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:02.798608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb229502-67d5-4954-9b5e-a9251c84f10d service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:02.798899 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb229502-67d5-4954-9b5e-a9251c84f10d service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:02.799183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb229502-67d5-4954-9b5e-a9251c84f10d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:02.801566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-bb229502-67d5-4954-9b5e-a9251c84f10d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:02.916014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb229502-67d5-4954-9b5e-a9251c84f10d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:02.917495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb229502-67d5-4954-9b5e-a9251c84f10d service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:02.937116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 720/1443] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:45:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:45:05.169172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376760-80d5-4dc6-b3e2-23233bd329a6 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:05.169774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376760-80d5-4dc6-b3e2-23233bd329a6 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:05.169774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db376760-80d5-4dc6-b3e2-23233bd329a6 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:05.171120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:45:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:45:05.176950 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfa12b-0c3a-4920-95ef-ca52b4e8d82c None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:05.177341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfa12b-0c3a-4920-95ef-ca52b4e8d82c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:05.177512 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dfa12b-0c3a-4920-95ef-ca52b4e8d82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:05.243242 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-06dfa12b-0c3a-4920-95ef-ca52b4e8d82c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:05.244502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-06dfa12b-0c3a-4920-95ef-ca52b4e8d82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:05.257926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:05.342580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:45:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:45:05.382239 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db376760-80d5-4dc6-b3e2-23233bd329a6 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:05.383810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-db376760-80d5-4dc6-b3e2-23233bd329a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:05.487733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db376760-80d5-4dc6-b3e2-23233bd329a6 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:05.500548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:05.565150 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224a55d1-e2a5-456a-b5ea-a219188c06c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:05.565787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a55d1-e2a5-456a-b5ea-a219188c06c6 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:05.566066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a55d1-e2a5-456a-b5ea-a219188c06c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:05.566338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224a55d1-e2a5-456a-b5ea-a219188c06c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:05.568618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-224a55d1-e2a5-456a-b5ea-a219188c06c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:05.680169 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a55d1-e2a5-456a-b5ea-a219188c06c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:05.681536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224a55d1-e2a5-456a-b5ea-a219188c06c6 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:05.699393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 725/1446] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:45:08.136211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe743d-735d-4896-85ae-3692b92b558f None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:08.136754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe743d-735d-4896-85ae-3692b92b558f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:08.136754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe743d-735d-4896-85ae-3692b92b558f None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:08.138144 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:45:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:45:08.143429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb841f7-f346-4712-bf04-917b069f77d3 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:08.143887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb841f7-f346-4712-bf04-917b069f77d3 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:08.144019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb841f7-f346-4712-bf04-917b069f77d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:08.191793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:08.213554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-2eb841f7-f346-4712-bf04-917b069f77d3 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:08.214792 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-2eb841f7-f346-4712-bf04-917b069f77d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:08.310496 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:45:08.351325 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe743d-735d-4896-85ae-3692b92b558f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:08.352895 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-36fe743d-735d-4896-85ae-3692b92b558f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:08.458048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe743d-735d-4896-85ae-3692b92b558f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:08.498902 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:08.533850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13a064-a442-454b-b7ec-ec1f732061d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:08.534514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13a064-a442-454b-b7ec-ec1f732061d6 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:08.534801 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13a064-a442-454b-b7ec-ec1f732061d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:08.535081 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13a064-a442-454b-b7ec-ec1f732061d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:08.536793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-dc13a064-a442-454b-b7ec-ec1f732061d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:08.646711 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13a064-a442-454b-b7ec-ec1f732061d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:08.648193 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc13a064-a442-454b-b7ec-ec1f732061d6 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:08.666865 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 723/1449] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:45:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:45:10.819416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5359969-20d2-4803-9e6c-eca9fc1a5c01 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:10.820011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5359969-20d2-4803-9e6c-eca9fc1a5c01 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:10.820011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5359969-20d2-4803-9e6c-eca9fc1a5c01 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:10.821399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:45:10.826776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a0532-a8b0-47c6-8f81-659a0d85539a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:10.827190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a0532-a8b0-47c6-8f81-659a0d85539a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:10.827336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a0532-a8b0-47c6-8f81-659a0d85539a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:10.894935 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-3f8a0532-a8b0-47c6-8f81-659a0d85539a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:10.896765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3f8a0532-a8b0-47c6-8f81-659a0d85539a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:10.917430 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:10.997841 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:45:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:45:11.037272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5359969-20d2-4803-9e6c-eca9fc1a5c01 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:11.038833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a5359969-20d2-4803-9e6c-eca9fc1a5c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:11.141117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5359969-20d2-4803-9e6c-eca9fc1a5c01 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:11.163117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:11.217810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9377b56-129d-4fbb-838e-0e1a8e7ce438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:11.218468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9377b56-129d-4fbb-838e-0e1a8e7ce438 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:11.218775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9377b56-129d-4fbb-838e-0e1a8e7ce438 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:11.219050 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9377b56-129d-4fbb-838e-0e1a8e7ce438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:11.220761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c9377b56-129d-4fbb-838e-0e1a8e7ce438 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:11.334466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9377b56-129d-4fbb-838e-0e1a8e7ce438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:11.335953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9377b56-129d-4fbb-838e-0e1a8e7ce438 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:11.354842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 728/1452] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:45:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:45:13.475741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a8eed-1332-4c79-98cf-120fd3153ad7 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:13.476376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a8eed-1332-4c79-98cf-120fd3153ad7 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:13.476376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30a8eed-1332-4c79-98cf-120fd3153ad7 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:13.477762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:45:13.483714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918d0fd-2c96-4b62-a50a-3ec9984fe2bf None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:13.484192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918d0fd-2c96-4b62-a50a-3ec9984fe2bf None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:13.484302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918d0fd-2c96-4b62-a50a-3ec9984fe2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:13.548784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:13.551661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-e918d0fd-2c96-4b62-a50a-3ec9984fe2bf None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:13.553520 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e918d0fd-2c96-4b62-a50a-3ec9984fe2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:13.651621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:45:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:45:13.691577 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a8eed-1332-4c79-98cf-120fd3153ad7 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:13.693785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f30a8eed-1332-4c79-98cf-120fd3153ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:13.798761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30a8eed-1332-4c79-98cf-120fd3153ad7 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:13.849803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:13.876098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce9c748-5e74-445d-aaed-4d74ac3e4d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:13.876753 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9c748-5e74-445d-aaed-4d74ac3e4d16 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:13.877043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9c748-5e74-445d-aaed-4d74ac3e4d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:13.877314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce9c748-5e74-445d-aaed-4d74ac3e4d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:13.878991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1ce9c748-5e74-445d-aaed-4d74ac3e4d16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:13.995607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9c748-5e74-445d-aaed-4d74ac3e4d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:13.997091 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce9c748-5e74-445d-aaed-4d74ac3e4d16 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:14.017292 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 726/1455] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:45:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:45:16.150420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5f69d-9c9b-441b-a050-1636e86a5fd5 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:16.150972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5f69d-9c9b-441b-a050-1636e86a5fd5 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:16.150972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f5f69d-9c9b-441b-a050-1636e86a5fd5 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:16.152684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:45:16.158872 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b63b6b-3893-4394-a8b6-b2ee1c5d71a4 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:16.159436 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b63b6b-3893-4394-a8b6-b2ee1c5d71a4 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:16.159577 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b63b6b-3893-4394-a8b6-b2ee1c5d71a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:16.233500 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f4b63b6b-3893-4394-a8b6-b2ee1c5d71a4 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:16.234723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f4b63b6b-3893-4394-a8b6-b2ee1c5d71a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:16.274383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:16.349656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:45:16.388829 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5f69d-9c9b-441b-a050-1636e86a5fd5 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:16.390346 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-50f5f69d-9c9b-441b-a050-1636e86a5fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:16.494538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f5f69d-9c9b-441b-a050-1636e86a5fd5 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:16.528572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:16.581904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb5188-68e9-48fe-a491-6aede68d1b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:16.582575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb5188-68e9-48fe-a491-6aede68d1b0c service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:16.582914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb5188-68e9-48fe-a491-6aede68d1b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:16.583201 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb5188-68e9-48fe-a491-6aede68d1b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:16.584958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-26bb5188-68e9-48fe-a491-6aede68d1b0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:16.702497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb5188-68e9-48fe-a491-6aede68d1b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:16.703980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb5188-68e9-48fe-a491-6aede68d1b0c service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:16.722475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 731/1458] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:45:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:45:18.891034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818bbc6-393e-4753-abdb-c93947113c9f None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:18.891605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818bbc6-393e-4753-abdb-c93947113c9f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:18.891605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d818bbc6-393e-4753-abdb-c93947113c9f None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:18.893034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:45:18.898399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd85930-00a1-42ba-b963-624710299b7f None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:18.898786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd85930-00a1-42ba-b963-624710299b7f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:18.898979 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd85930-00a1-42ba-b963-624710299b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:18.967535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5cd85930-00a1-42ba-b963-624710299b7f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:18.968794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5cd85930-00a1-42ba-b963-624710299b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:18.977242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:19.066762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:45:19.105337 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818bbc6-393e-4753-abdb-c93947113c9f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:19.106883 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d818bbc6-393e-4753-abdb-c93947113c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:19.221667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d818bbc6-393e-4753-abdb-c93947113c9f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:19.279546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:19.299737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc577b8f-5ef6-41d8-b497-379cd03e1e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:19.300443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc577b8f-5ef6-41d8-b497-379cd03e1e97 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:19.300733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc577b8f-5ef6-41d8-b497-379cd03e1e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:19.301016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc577b8f-5ef6-41d8-b497-379cd03e1e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:19.303425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-fc577b8f-5ef6-41d8-b497-379cd03e1e97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:19.424661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc577b8f-5ef6-41d8-b497-379cd03e1e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:19.427602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc577b8f-5ef6-41d8-b497-379cd03e1e97 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:19.448343 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 729/1461] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:45:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:45:21.608056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76144e11-ea02-4308-abdd-58eb072d0387 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:21.608678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76144e11-ea02-4308-abdd-58eb072d0387 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:21.608678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76144e11-ea02-4308-abdd-58eb072d0387 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:21.610113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:45:21.616235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5ff9e-349c-41cb-a60e-1e46c94ccfb9 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:21.616703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5ff9e-349c-41cb-a60e-1e46c94ccfb9 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:21.616810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a5ff9e-349c-41cb-a60e-1e46c94ccfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:21.681207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-84a5ff9e-349c-41cb-a60e-1e46c94ccfb9 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:21.682415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-84a5ff9e-349c-41cb-a60e-1e46c94ccfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:21.722567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:21.788843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:45:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:45:21.826303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76144e11-ea02-4308-abdd-58eb072d0387 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:21.827837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-76144e11-ea02-4308-abdd-58eb072d0387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:21.932304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76144e11-ea02-4308-abdd-58eb072d0387 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:21.970001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:22.005973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88249e-dbfa-4cd1-90a1-ae77de95e1e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:22.006541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88249e-dbfa-4cd1-90a1-ae77de95e1e5 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:22.006827 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88249e-dbfa-4cd1-90a1-ae77de95e1e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:22.007104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88249e-dbfa-4cd1-90a1-ae77de95e1e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:22.008795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ea88249e-dbfa-4cd1-90a1-ae77de95e1e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:22.122204 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88249e-dbfa-4cd1-90a1-ae77de95e1e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:22.123663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea88249e-dbfa-4cd1-90a1-ae77de95e1e5 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:22.142478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 734/1464] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:45:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:45:24.214757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c93e19-7f1f-47ea-8f7a-de9b1e92eb92 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:24.215404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c93e19-7f1f-47ea-8f7a-de9b1e92eb92 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:24.215544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c93e19-7f1f-47ea-8f7a-de9b1e92eb92 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:24.216987 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:45:24.222559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29ca48-406d-4d89-8b00-5710706ccfe3 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:24.222967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29ca48-406d-4d89-8b00-5710706ccfe3 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:24.223132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29ca48-406d-4d89-8b00-5710706ccfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:24.287916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-aa29ca48-406d-4d89-8b00-5710706ccfe3 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:24.289174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-aa29ca48-406d-4d89-8b00-5710706ccfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:24.307534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:24.390681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:45:24.428059 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c93e19-7f1f-47ea-8f7a-de9b1e92eb92 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:24.429669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-38c93e19-7f1f-47ea-8f7a-de9b1e92eb92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:24.537217 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c93e19-7f1f-47ea-8f7a-de9b1e92eb92 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:24.606399 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:24.613863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfa971-c11b-4ccd-9974-f4502febdc24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:24.614519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfa971-c11b-4ccd-9974-f4502febdc24 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:24.615340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfa971-c11b-4ccd-9974-f4502febdc24 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:24.615623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfa971-c11b-4ccd-9974-f4502febdc24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:24.617329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-35dfa971-c11b-4ccd-9974-f4502febdc24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:24.725025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfa971-c11b-4ccd-9974-f4502febdc24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:24.726458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfa971-c11b-4ccd-9974-f4502febdc24 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:24.745370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 732/1467] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:45:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:45:26.802618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb1b98-b04a-4d96-a889-b4fe95745b47 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:26.803242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb1b98-b04a-4d96-a889-b4fe95745b47 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:26.803242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdb1b98-b04a-4d96-a889-b4fe95745b47 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:26.804655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:45:26.810663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b8a05-620c-4b68-90d9-0a165b27c02a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:26.811256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b8a05-620c-4b68-90d9-0a165b27c02a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:26.811256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b8a05-620c-4b68-90d9-0a165b27c02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:26.876464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-9e5b8a05-620c-4b68-90d9-0a165b27c02a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:26.877690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9e5b8a05-620c-4b68-90d9-0a165b27c02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:26.925742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:26.974802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:45:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:45:27.012419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb1b98-b04a-4d96-a889-b4fe95745b47 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:27.013948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-acdb1b98-b04a-4d96-a889-b4fe95745b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:27.114526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdb1b98-b04a-4d96-a889-b4fe95745b47 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:27.161796 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:27.188537 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b42c781-6f0e-44b5-82fe-f25a58afa282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:27.189193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42c781-6f0e-44b5-82fe-f25a58afa282 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:27.189486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42c781-6f0e-44b5-82fe-f25a58afa282 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:27.189766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b42c781-6f0e-44b5-82fe-f25a58afa282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:27.191420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9b42c781-6f0e-44b5-82fe-f25a58afa282 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:27.300421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42c781-6f0e-44b5-82fe-f25a58afa282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:27.301875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b42c781-6f0e-44b5-82fe-f25a58afa282 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:27.320950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 737/1470] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:45:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:45:36.972985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=22 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:37.318127 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=66 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:47.650926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2ec46-e7a1-4de2-ac7a-6ec90a8db1d9 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:47.651528 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2ec46-e7a1-4de2-ac7a-6ec90a8db1d9 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:47.651528 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2ec46-e7a1-4de2-ac7a-6ec90a8db1d9 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:47.653032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 734/1471] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:45:47.658800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b761ce-cfeb-47e9-bb12-322af982ec43 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:47.659333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b761ce-cfeb-47e9-bb12-322af982ec43 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:47.659438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b761ce-cfeb-47e9-bb12-322af982ec43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:47.761494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-72b761ce-cfeb-47e9-bb12-322af982ec43 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:47.763318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-72b761ce-cfeb-47e9-bb12-322af982ec43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:47.889203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 738/1472] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:45:47.982172 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6edb1-43a7-4983-880c-64e885ceba82 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:47.982459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6edb1-43a7-4983-880c-64e885ceba82 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:47.982741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef6edb1-43a7-4983-880c-64e885ceba82 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:47.984190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 735/1473] 10.222.0.64 () {54 vars in 962 bytes} [Thu Jul 14 17:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:45:47.989538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d039377-f62e-48f4-82c7-3f27e2b396cd None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:47.989897 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d039377-f62e-48f4-82c7-3f27e2b396cd None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:47.990243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d039377-f62e-48f4-82c7-3f27e2b396cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:48.070962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-3d039377-f62e-48f4-82c7-3f27e2b396cd None None] MFA Rules not processed for user `b2b4bf2aa1744a46b6d2a1b5a136a0e1`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:48.072211 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3d039377-f62e-48f4-82c7-3f27e2b396cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:48.093956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:48.155523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 739/1474] 10.222.0.64 () {58 vars in 1059 bytes} [Thu Jul 14 17:45:47 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:45:48.163852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6edb1-43a7-4983-880c-64e885ceba82 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:48.165503 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5ef6edb1-43a7-4983-880c-64e885ceba82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:48.278495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef6edb1-43a7-4983-880c-64e885ceba82 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:48.360002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b8b78-c61f-4b0c-b50f-e8232af43914 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2b4bf2aa1744a46b6d2a1b5a136a0e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:48.360655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b8b78-c61f-4b0c-b50f-e8232af43914 service glance] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:48.360936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b8b78-c61f-4b0c-b50f-e8232af43914 service glance] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:48.361209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b8b78-c61f-4b0c-b50f-e8232af43914 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:48.362873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-275b8b78-c61f-4b0c-b50f-e8232af43914 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:48.425564 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:48.525520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b8b78-c61f-4b0c-b50f-e8232af43914 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:48.528338 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b8b78-c61f-4b0c-b50f-e8232af43914 service glance] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:48.562966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 736/1475] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:45:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:45:50.764522 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63387387-3327-4b1f-b661-6bcb719193c2 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:50.765096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63387387-3327-4b1f-b661-6bcb719193c2 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:50.765196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63387387-3327-4b1f-b661-6bcb719193c2 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:50.766774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 740/1476] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:45:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:45:50.772365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182286da-4693-41ca-85c5-f05a3fcc2c8f None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:50.773691 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182286da-4693-41ca-85c5-f05a3fcc2c8f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:50.773691 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182286da-4693-41ca-85c5-f05a3fcc2c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:50.854979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-182286da-4693-41ca-85c5-f05a3fcc2c8f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:50.856278 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-182286da-4693-41ca-85c5-f05a3fcc2c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:50.967652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 737/1477] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:45:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:45:50.992905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63387387-3327-4b1f-b661-6bcb719193c2 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:50.994444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-63387387-3327-4b1f-b661-6bcb719193c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:51.187645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63387387-3327-4b1f-b661-6bcb719193c2 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:51.245151 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:51.293266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446dd966-413b-4df6-b044-7cab77d6402e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:51.293932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dd966-413b-4df6-b044-7cab77d6402e admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:51.294221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dd966-413b-4df6-b044-7cab77d6402e admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:51.294495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446dd966-413b-4df6-b044-7cab77d6402e admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:51.295236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446dd966-413b-4df6-b044-7cab77d6402e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:51.297136 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: WARNING py.warnings [None req-446dd966-413b-4df6-b044-7cab77d6402e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:45:51.297136 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: warnings.warn(msg) Jul 14 17:45:51.297136 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: Jul 14 17:45:51.298286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446dd966-413b-4df6-b044-7cab77d6402e admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:51.318097 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 741/1478] 10.222.0.64 () {56 vars in 1108 bytes} [Thu Jul 14 17:45:50 2022] GET /identity/v3/projects => generated 2553 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:45:53.432472 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a6259-d803-4155-92d4-bac0889a1f1f None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:53.433147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a6259-d803-4155-92d4-bac0889a1f1f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:53.433147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a6259-d803-4155-92d4-bac0889a1f1f None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:53.434443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 738/1479] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:45:53.440465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f0eba-6b7f-4df9-9090-8e516098b36a None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:53.441180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f0eba-6b7f-4df9-9090-8e516098b36a None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:53.441180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f0eba-6b7f-4df9-9090-8e516098b36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:53.515156 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-a29f0eba-6b7f-4df9-9090-8e516098b36a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:53.516465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a29f0eba-6b7f-4df9-9090-8e516098b36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:53.624914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 742/1480] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:45:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:45:53.670635 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a6259-d803-4155-92d4-bac0889a1f1f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:53.672333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-832a6259-d803-4155-92d4-bac0889a1f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:53.683431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:53.836898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a6259-d803-4155-92d4-bac0889a1f1f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:53.926260 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:53.961997 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c9da7-580f-4d1f-a828-89f6d6436264 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:53.962662 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c9da7-580f-4d1f-a828-89f6d6436264 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:53.962948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c9da7-580f-4d1f-a828-89f6d6436264 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:53.963228 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c9da7-580f-4d1f-a828-89f6d6436264 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:53.964951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-076c9da7-580f-4d1f-a828-89f6d6436264 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:54.058889 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c9da7-580f-4d1f-a828-89f6d6436264 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:54.060563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c9da7-580f-4d1f-a828-89f6d6436264 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:54.080029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 739/1481] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:45:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:45:56.356064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feab069-782b-4cb1-be2e-c262660b0b85 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:56.356760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feab069-782b-4cb1-be2e-c262660b0b85 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:56.356760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feab069-782b-4cb1-be2e-c262660b0b85 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:56.358021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 743/1482] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:45:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:45:56.363542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22e54b-b477-4754-b2f4-4d6dc1746590 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:56.364156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22e54b-b477-4754-b2f4-4d6dc1746590 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:56.364156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a22e54b-b477-4754-b2f4-4d6dc1746590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:56.440957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-6a22e54b-b477-4754-b2f4-4d6dc1746590 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:56.442735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6a22e54b-b477-4754-b2f4-4d6dc1746590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:56.509655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:56.581427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 740/1483] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:45:56.618886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feab069-782b-4cb1-be2e-c262660b0b85 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:56.620523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-8feab069-782b-4cb1-be2e-c262660b0b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:56.646998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:56.735743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feab069-782b-4cb1-be2e-c262660b0b85 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:56.821645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961780cb-ab92-4098-aba0-39ab9542309b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:56.822298 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961780cb-ab92-4098-aba0-39ab9542309b service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:56.822580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961780cb-ab92-4098-aba0-39ab9542309b service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:56.822849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961780cb-ab92-4098-aba0-39ab9542309b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:56.824528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-961780cb-ab92-4098-aba0-39ab9542309b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:56.900647 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:56.935383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961780cb-ab92-4098-aba0-39ab9542309b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:56.936846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961780cb-ab92-4098-aba0-39ab9542309b service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:56.954893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 744/1484] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:45:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:45:59.132427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56e1f0-9259-48b6-b00b-9930b131a441 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:59.133682 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56e1f0-9259-48b6-b00b-9930b131a441 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:59.133682 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f56e1f0-9259-48b6-b00b-9930b131a441 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:59.134930 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 741/1485] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:45:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:45:59.140636 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c4ffa-d77d-420b-b7e2-0a1cf20c9ddc None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:59.141166 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c4ffa-d77d-420b-b7e2-0a1cf20c9ddc None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:59.141311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c4ffa-d77d-420b-b7e2-0a1cf20c9ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:59.211879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-680c4ffa-d77d-420b-b7e2-0a1cf20c9ddc None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:45:59.213151 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-680c4ffa-d77d-420b-b7e2-0a1cf20c9ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:59.329371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 745/1486] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:45:59.369569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56e1f0-9259-48b6-b00b-9930b131a441 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:45:59.371168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9f56e1f0-9259-48b6-b00b-9930b131a441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:59.493777 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f56e1f0-9259-48b6-b00b-9930b131a441 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:45:59.570652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:45:59.572479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcceaef7-0a12-48df-b00e-e716a458d5d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:45:59.573331 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceaef7-0a12-48df-b00e-e716a458d5d2 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:45:59.573927 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceaef7-0a12-48df-b00e-e716a458d5d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:45:59.574288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcceaef7-0a12-48df-b00e-e716a458d5d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:45:59.576512 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-fcceaef7-0a12-48df-b00e-e716a458d5d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:45:59.700392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcceaef7-0a12-48df-b00e-e716a458d5d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:45:59.701830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcceaef7-0a12-48df-b00e-e716a458d5d2 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:45:59.721132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 742/1487] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:45:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:46:01.952743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30eaf06-ba3a-4359-816c-d80908649049 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:46:01.953602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30eaf06-ba3a-4359-816c-d80908649049 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:46:01.953789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30eaf06-ba3a-4359-816c-d80908649049 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:46:01.955208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 746/1488] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:46:01.961924 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0a891-41ed-4992-9d47-9c9fba969b70 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:46:01.962489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0a891-41ed-4992-9d47-9c9fba969b70 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:46:01.962913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe0a891-41ed-4992-9d47-9c9fba969b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:46:02.034543 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-9fe0a891-41ed-4992-9d47-9c9fba969b70 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:46:02.035784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9fe0a891-41ed-4992-9d47-9c9fba969b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:46:02.094674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:46:02.147546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 743/1489] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:46:02.191517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30eaf06-ba3a-4359-816c-d80908649049 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:46:02.193879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a30eaf06-ba3a-4359-816c-d80908649049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:46:02.244972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:46:02.319791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30eaf06-ba3a-4359-816c-d80908649049 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:46:02.402190 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ce3e0-5c9c-4be2-857c-dcc985ce4c0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:46:02.402875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce3e0-5c9c-4be2-857c-dcc985ce4c0d service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:46:02.403216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce3e0-5c9c-4be2-857c-dcc985ce4c0d service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:46:02.403516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ce3e0-5c9c-4be2-857c-dcc985ce4c0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:46:02.405355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-841ce3e0-5c9c-4be2-857c-dcc985ce4c0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:46:02.498784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:46:02.531913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ce3e0-5c9c-4be2-857c-dcc985ce4c0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:46:02.533399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ce3e0-5c9c-4be2-857c-dcc985ce4c0d service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:46:02.552357 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 747/1490] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:46:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:46:04.885101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1da1-0112-4250-8398-2ee693989ee5 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:46:04.885729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1da1-0112-4250-8398-2ee693989ee5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:46:04.885729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446d1da1-0112-4250-8398-2ee693989ee5 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:46:04.887045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 744/1491] 10.222.0.64 () {54 vars in 926 bytes} [Thu Jul 14 17:46:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:46:04.894004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529055e-df89-49be-8e65-a4fdc01f78d1 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:46:04.894508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529055e-df89-49be-8e65-a4fdc01f78d1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:46:04.894881 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529055e-df89-49be-8e65-a4fdc01f78d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:46:04.967221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-2529055e-df89-49be-8e65-a4fdc01f78d1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:46:04.968495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-2529055e-df89-49be-8e65-a4fdc01f78d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:46:05.073739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 748/1492] 10.222.0.64 () {58 vars in 1023 bytes} [Thu Jul 14 17:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:46:05.114014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d1da1-0112-4250-8398-2ee693989ee5 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:46:05.115883 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-446d1da1-0112-4250-8398-2ee693989ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:46:05.231428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446d1da1-0112-4250-8398-2ee693989ee5 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:46:05.310204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5bff21-6d33-483e-8925-da8d2a180819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:46:05.310856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bff21-6d33-483e-8925-da8d2a180819 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:46:05.311140 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bff21-6d33-483e-8925-da8d2a180819 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:46:05.311417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bff21-6d33-483e-8925-da8d2a180819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:46:05.313111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5b5bff21-6d33-483e-8925-da8d2a180819 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:46:05.323653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:46:05.465648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5bff21-6d33-483e-8925-da8d2a180819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:46:05.467096 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5bff21-6d33-483e-8925-da8d2a180819 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:46:05.493627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 745/1493] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:46:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:46:15.071839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=82 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:46:15.489198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=50 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:48:09.400276 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3739d3-53d7-4204-aada-025dd3a2de22 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:09.400871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3739d3-53d7-4204-aada-025dd3a2de22 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:09.400871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3739d3-53d7-4204-aada-025dd3a2de22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:09.492841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-bb3739d3-53d7-4204-aada-025dd3a2de22 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:48:09.494102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-bb3739d3-53d7-4204-aada-025dd3a2de22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:48:09.613770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 749/1494] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:48:09.622093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a22b80-5010-4ff6-89cb-dfd8803cd7ca None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:09.622961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a22b80-5010-4ff6-89cb-dfd8803cd7ca None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:09.622961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a22b80-5010-4ff6-89cb-dfd8803cd7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:09.648324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-d1a22b80-5010-4ff6-89cb-dfd8803cd7ca None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:48:09.649567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d1a22b80-5010-4ff6-89cb-dfd8803cd7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:48:09.660138 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 746/1495] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:48:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:48:09.664772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3739d3-53d7-4204-aada-025dd3a2de22 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:48:09.675565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3739d3-53d7-4204-aada-025dd3a2de22 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:48:09.680264 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02343b-938f-44e2-87e4-99f1b8e8f209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:48:09.680935 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02343b-938f-44e2-87e4-99f1b8e8f209 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:09.681222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02343b-938f-44e2-87e4-99f1b8e8f209 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:09.681500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02343b-938f-44e2-87e4-99f1b8e8f209 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:09.682242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02343b-938f-44e2-87e4-99f1b8e8f209 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:48:09.682551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02343b-938f-44e2-87e4-99f1b8e8f209 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:48:09.683578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: WARNING py.warnings [None req-4c02343b-938f-44e2-87e4-99f1b8e8f209 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:48:09.683578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: warnings.warn(msg) Jul 14 17:48:09.683578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: Jul 14 17:48:09.684504 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02343b-938f-44e2-87e4-99f1b8e8f209 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:48:09.697440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 750/1496] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:48:09.705716 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a22b80-5010-4ff6-89cb-dfd8803cd7ca None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:48:09.715670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a22b80-5010-4ff6-89cb-dfd8803cd7ca None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:48:09.721148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1a6d7-25ad-4b89-9c91-0a96a017aacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:48:09.721814 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1a6d7-25ad-4b89-9c91-0a96a017aacf admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:09.722100 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1a6d7-25ad-4b89-9c91-0a96a017aacf admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:09.722409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e1a6d7-25ad-4b89-9c91-0a96a017aacf admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:09.723101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1a6d7-25ad-4b89-9c91-0a96a017aacf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:48:09.724945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: WARNING py.warnings [None req-71e1a6d7-25ad-4b89-9c91-0a96a017aacf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:48:09.724945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: warnings.warn(msg) Jul 14 17:48:09.724945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: Jul 14 17:48:09.726159 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e1a6d7-25ad-4b89-9c91-0a96a017aacf admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:48:09.763497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 747/1497] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:48:09 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:48:09.768357 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02343b-938f-44e2-87e4-99f1b8e8f209 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:48:09.780199 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02343b-938f-44e2-87e4-99f1b8e8f209 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:48:09.838202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:48:09.848754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991fd9c-f20e-436e-bf1d-a630fc1e319b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:48:09.849676 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991fd9c-f20e-436e-bf1d-a630fc1e319b admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:09.850178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991fd9c-f20e-436e-bf1d-a630fc1e319b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:09.850516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991fd9c-f20e-436e-bf1d-a630fc1e319b admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:09.851404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991fd9c-f20e-436e-bf1d-a630fc1e319b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:48:09.852554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: WARNING py.warnings [None req-a991fd9c-f20e-436e-bf1d-a630fc1e319b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:48:09.852554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: warnings.warn(msg) Jul 14 17:48:09.852554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: Jul 14 17:48:09.853908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991fd9c-f20e-436e-bf1d-a630fc1e319b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:48:09.874854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 751/1498] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:48:09 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:48:09.879239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1a6d7-25ad-4b89-9c91-0a96a017aacf admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:48:09.892105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e1a6d7-25ad-4b89-9c91-0a96a017aacf admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:48:09.897300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa17b01-bb8a-454c-9c25-31d9eb5a0f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:48:09.897973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa17b01-bb8a-454c-9c25-31d9eb5a0f92 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:09.898256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa17b01-bb8a-454c-9c25-31d9eb5a0f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:09.898531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa17b01-bb8a-454c-9c25-31d9eb5a0f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:09.899259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa17b01-bb8a-454c-9c25-31d9eb5a0f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:48:09.900304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: WARNING py.warnings [None req-3fa17b01-bb8a-454c-9c25-31d9eb5a0f92 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:48:09.900304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: warnings.warn(msg) Jul 14 17:48:09.900304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: Jul 14 17:48:09.901183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa17b01-bb8a-454c-9c25-31d9eb5a0f92 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:48:09.915292 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 748/1499] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:48:09.921568 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991fd9c-f20e-436e-bf1d-a630fc1e319b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:48:09.931600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991fd9c-f20e-436e-bf1d-a630fc1e319b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:48:09.936302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dba00a1-d73a-4825-9b25-02493b753fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:48:09.936980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba00a1-d73a-4825-9b25-02493b753fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:09.937265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba00a1-d73a-4825-9b25-02493b753fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:09.937540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dba00a1-d73a-4825-9b25-02493b753fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:09.938286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba00a1-d73a-4825-9b25-02493b753fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:48:09.939320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: WARNING py.warnings [None req-7dba00a1-d73a-4825-9b25-02493b753fa3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:48:09.939320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: warnings.warn(msg) Jul 14 17:48:09.939320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: Jul 14 17:48:09.940712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dba00a1-d73a-4825-9b25-02493b753fa3 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:48:09.954042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 752/1500] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:48:09.958853 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa17b01-bb8a-454c-9c25-31d9eb5a0f92 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:48:09.971324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa17b01-bb8a-454c-9c25-31d9eb5a0f92 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:48:09.976224 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ebac65-d280-4c09-96fe-be391431b3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:48:09.977129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebac65-d280-4c09-96fe-be391431b3ee admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:09.977414 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebac65-d280-4c09-96fe-be391431b3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:09.977692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ebac65-d280-4c09-96fe-be391431b3ee admin admin] PATH_INFO: `/v3/projects/9e1dbd38a2194268b38528bac0e52d52/users/33159a78831d41b884fa28fd0f3046ce/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:09.994179 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebac65-d280-4c09-96fe-be391431b3ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e1dbd38a2194268b38528bac0e52d52, user_id=33159a78831d41b884fa28fd0f3046ce, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:48:09.995308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: WARNING py.warnings [None req-d5ebac65-d280-4c09-96fe-be391431b3ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:48:09.995308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: warnings.warn(msg) Jul 14 17:48:09.995308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: Jul 14 17:48:09.997275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ebac65-d280-4c09-96fe-be391431b3ee admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:48:10.013156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 749/1501] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:48:09 2022] PUT /identity/v3/projects/9e1dbd38a2194268b38528bac0e52d52/users/33159a78831d41b884fa28fd0f3046ce/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:48:10.025823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717f14c-027c-4440-8dbf-f325169e5838 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:10.026110 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717f14c-027c-4440-8dbf-f325169e5838 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:10.026394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717f14c-027c-4440-8dbf-f325169e5838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:10.054389 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-b717f14c-027c-4440-8dbf-f325169e5838 None None] MFA Rules not processed for user `33159a78831d41b884fa28fd0f3046ce`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:48:10.055618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b717f14c-027c-4440-8dbf-f325169e5838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:48:10.122543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 753/1502] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:48:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:48:10.136326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27169c-56ba-4e32-928e-03043903d7e1 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:10.136618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27169c-56ba-4e32-928e-03043903d7e1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:10.136902 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e27169c-56ba-4e32-928e-03043903d7e1 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:10.138299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 750/1503] 10.222.0.64 () {54 vars in 963 bytes} [Thu Jul 14 17:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:48:10.144008 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe40a7-2eee-4063-92b1-4036bb57fd76 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:10.144308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe40a7-2eee-4063-92b1-4036bb57fd76 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:10.144591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe40a7-2eee-4063-92b1-4036bb57fd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:10.205227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-0dbe40a7-2eee-4063-92b1-4036bb57fd76 None None] MFA Rules not processed for user `e49fad7db3c643b99e9071d9ca5f0196`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:48:10.207069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0dbe40a7-2eee-4063-92b1-4036bb57fd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:48:10.281898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 754/1504] 10.222.0.64 () {58 vars in 1060 bytes} [Thu Jul 14 17:48:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:48:10.290157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27169c-56ba-4e32-928e-03043903d7e1 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:48:10.301816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e27169c-56ba-4e32-928e-03043903d7e1 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:48:10.305426 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aded04-7b1f-4913-8083-6d27b978f186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:48:10.306072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aded04-7b1f-4913-8083-6d27b978f186 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:10.306358 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aded04-7b1f-4913-8083-6d27b978f186 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:10.306638 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aded04-7b1f-4913-8083-6d27b978f186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:10.315133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aded04-7b1f-4913-8083-6d27b978f186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:48:10.316613 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aded04-7b1f-4913-8083-6d27b978f186 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:48:10.325694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 751/1505] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:48:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:48:10.368978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c563c-3afb-462b-a24f-9cfa24657878 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:10.369275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c563c-3afb-462b-a24f-9cfa24657878 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:10.369554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942c563c-3afb-462b-a24f-9cfa24657878 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:10.370912 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 755/1506] 10.222.0.64 () {52 vars in 829 bytes} [Thu Jul 14 17:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:48:10.374605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aded04-7b1f-4913-8083-6d27b978f186 service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:48:10.436627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aded04-7b1f-4913-8083-6d27b978f186 service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:48:10.441278 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:48:10.442158 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:10.442444 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:10.442720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] PATH_INFO: `/v3/users/33159a78831d41b884fa28fd0f3046ce` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:10.443994 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] RBAC: Authorizing `identity:delete_user(user_id=33159a78831d41b884fa28fd0f3046ce)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:48:10.445023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: WARNING py.warnings [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:48:10.445023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: warnings.warn(msg) Jul 14 17:48:10.445023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: Jul 14 17:48:10.446229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:48:10.512218 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33159a78831d41b884fa28fd0f3046ce'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:48:10.530578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33159a78831d41b884fa28fd0f3046ce'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:48:10.540922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33159a78831d41b884fa28fd0f3046ce'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:48:10.542005 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33159a78831d41b884fa28fd0f3046ce'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:48:10.550912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:48:10.556250 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33159a78831d41b884fa28fd0f3046ce'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:48:10.563648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 752/1507] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:48:10 2022] DELETE /identity/v3/users/33159a78831d41b884fa28fd0f3046ce => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:48:10.568347 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247ccfd-f80f-4d6a-b5d6-d73053271a82 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:10.568988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247ccfd-f80f-4d6a-b5d6-d73053271a82 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:10.568988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7247ccfd-f80f-4d6a-b5d6-d73053271a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:10.587130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-7247ccfd-f80f-4d6a-b5d6-d73053271a82 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:48:10.588371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7247ccfd-f80f-4d6a-b5d6-d73053271a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:48:10.601088 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:48:10.650055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 756/1508] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:48:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:48:10.660210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:48:10.661644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:48:10.747646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241862ac-32b6-4900-8d6a-6da0f9ed589f admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:48:10.751363 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f2345-4b79-4c4e-8f7e-39b8b7c122a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:48:10.752112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f2345-4b79-4c4e-8f7e-39b8b7c122a0 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:10.752404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f2345-4b79-4c4e-8f7e-39b8b7c122a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:10.752688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f2345-4b79-4c4e-8f7e-39b8b7c122a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:10.761973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f2345-4b79-4c4e-8f7e-39b8b7c122a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:48:10.763476 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f2345-4b79-4c4e-8f7e-39b8b7c122a0 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:48:10.773718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 753/1509] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:48:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:48:11.104145 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247ccfd-f80f-4d6a-b5d6-d73053271a82 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:48:11.105734 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7247ccfd-f80f-4d6a-b5d6-d73053271a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:48:11.122068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7247ccfd-f80f-4d6a-b5d6-d73053271a82 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:48:11.128284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48872eb-384c-40c7-a1c4-fcf0595720a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:48:11.129216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48872eb-384c-40c7-a1c4-fcf0595720a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:48:11.129469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48872eb-384c-40c7-a1c4-fcf0595720a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:48:11.129747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48872eb-384c-40c7-a1c4-fcf0595720a9 admin admin] PATH_INFO: `/v3/projects/9e1dbd38a2194268b38528bac0e52d52` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:48:11.131176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48872eb-384c-40c7-a1c4-fcf0595720a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e1dbd38a2194268b38528bac0e52d52)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:48:11.132249 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: WARNING py.warnings [None req-e48872eb-384c-40c7-a1c4-fcf0595720a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:48:11.132249 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: warnings.warn(msg) Jul 14 17:48:11.132249 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: Jul 14 17:48:11.133466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48872eb-384c-40c7-a1c4-fcf0595720a9 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:48:11.198876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e48872eb-384c-40c7-a1c4-fcf0595720a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e1dbd38a2194268b38528bac0e52d52', 'request_id': 'req-e48872eb-384c-40c7-a1c4-fcf0595720a9'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:48:11.210642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e48872eb-384c-40c7-a1c4-fcf0595720a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e1dbd38a2194268b38528bac0e52d52', 'request_id': 'req-e48872eb-384c-40c7-a1c4-fcf0595720a9'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:48:11.216871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e48872eb-384c-40c7-a1c4-fcf0595720a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e1dbd38a2194268b38528bac0e52d52', 'request_id': 'req-e48872eb-384c-40c7-a1c4-fcf0595720a9'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:48:11.235538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e48872eb-384c-40c7-a1c4-fcf0595720a9 admin admin] The token cache is being invalidate because project 9e1dbd38a2194268b38528bac0e52d52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:48:11.235831 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e48872eb-384c-40c7-a1c4-fcf0595720a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:48:11.237139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e48872eb-384c-40c7-a1c4-fcf0595720a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:48:11.238812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 757/1510] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:48:11 2022] DELETE /identity/v3/projects/9e1dbd38a2194268b38528bac0e52d52 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:48:20.769384 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=38 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:48:21.232024 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:49:47.642273 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d9c2c-5ad0-4a5a-b270-8bdb38151738 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:49:47.642912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d9c2c-5ad0-4a5a-b270-8bdb38151738 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:49:47.642912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d9c2c-5ad0-4a5a-b270-8bdb38151738 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:49:47.644276 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 17:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:49:47.649139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d531a-52b1-4481-98fa-9c9bb5ea6fc6 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:49:47.649589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d531a-52b1-4481-98fa-9c9bb5ea6fc6 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:49:47.649733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d531a-52b1-4481-98fa-9c9bb5ea6fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:49:47.668333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4d8d531a-52b1-4481-98fa-9c9bb5ea6fc6 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:49:47.669666 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4d8d531a-52b1-4481-98fa-9c9bb5ea6fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:49:47.731616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 17:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:49:47.886304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d9c2c-5ad0-4a5a-b270-8bdb38151738 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:49:47.888508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-721d9c2c-5ad0-4a5a-b270-8bdb38151738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:49:47.997530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d9c2c-5ad0-4a5a-b270-8bdb38151738 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:49:48.003002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0f54b-7e5a-4913-a3ce-5f04d2acc544 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:49:48.003654 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0f54b-7e5a-4913-a3ce-5f04d2acc544 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:49:48.003956 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0f54b-7e5a-4913-a3ce-5f04d2acc544 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:49:48.004244 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0f54b-7e5a-4913-a3ce-5f04d2acc544 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:49:48.012865 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0f54b-7e5a-4913-a3ce-5f04d2acc544 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:49:48.014458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0f54b-7e5a-4913-a3ce-5f04d2acc544 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:49:48.023608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 755/1513] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:49:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:49:48.853555 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81697d77-b0f0-465b-ac1e-01521bdfd5e0 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:49:48.854102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81697d77-b0f0-465b-ac1e-01521bdfd5e0 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:49:48.854218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81697d77-b0f0-465b-ac1e-01521bdfd5e0 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:49:48.856237 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 17:49:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:49:48.861253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b51174-d436-4d88-9b70-ebea79fca1a6 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:49:48.861691 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b51174-d436-4d88-9b70-ebea79fca1a6 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:49:48.861822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b51174-d436-4d88-9b70-ebea79fca1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:49:48.879766 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-e3b51174-d436-4d88-9b70-ebea79fca1a6 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:49:48.881009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e3b51174-d436-4d88-9b70-ebea79fca1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:49:48.888824 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 17:49:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:49:49.040341 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81697d77-b0f0-465b-ac1e-01521bdfd5e0 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:49:49.055320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81697d77-b0f0-465b-ac1e-01521bdfd5e0 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:49:49.060722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d37b77-5748-4a80-8750-e61a89660a6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:49:49.061364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d37b77-5748-4a80-8750-e61a89660a6e service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:49:49.061649 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d37b77-5748-4a80-8750-e61a89660a6e service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:49:49.061933 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d37b77-5748-4a80-8750-e61a89660a6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:49:49.070018 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d37b77-5748-4a80-8750-e61a89660a6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:49:49.071594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d37b77-5748-4a80-8750-e61a89660a6e service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:49:49.081577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 760/1516] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:49:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:49:50.065551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd53c9-3551-4162-a37c-a1028bd7fbe8 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:49:50.066142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd53c9-3551-4162-a37c-a1028bd7fbe8 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:49:50.066142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd53c9-3551-4162-a37c-a1028bd7fbe8 None None] PATH_INFO: `/v3` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:49:50.067430 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 757/1517] 10.222.0.57 () {54 vars in 925 bytes} [Thu Jul 14 17:49:50 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:49:50.073865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc65577-77d7-479a-b12f-ceb34a596539 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:49:50.074349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc65577-77d7-479a-b12f-ceb34a596539 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:49:50.074498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc65577-77d7-479a-b12f-ceb34a596539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:49:50.123569 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-cfc65577-77d7-479a-b12f-ceb34a596539 None None] MFA Rules not processed for user `619e3bec841343c5ad144f1d4e4d436d`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:49:50.124830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-cfc65577-77d7-479a-b12f-ceb34a596539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:49:50.197824 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 761/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Thu Jul 14 17:49:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:49:58.875041 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=49 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:49:59.304374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9741a4-2b78-4995-93b4-af4369c34d05 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:49:59.304668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9741a4-2b78-4995-93b4-af4369c34d05 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:49:59.304954 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9741a4-2b78-4995-93b4-af4369c34d05 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:49:59.306376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 758/1519] 10.222.0.64 () {54 vars in 960 bytes} [Thu Jul 14 17:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:49:59.311778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbea81-ef84-44a4-8334-89d02330213d None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:49:59.312297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbea81-ef84-44a4-8334-89d02330213d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:49:59.312413 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bbea81-ef84-44a4-8334-89d02330213d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:49:59.330981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-72bbea81-ef84-44a4-8334-89d02330213d None None] MFA Rules not processed for user `619e3bec841343c5ad144f1d4e4d436d`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:49:59.332218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-72bbea81-ef84-44a4-8334-89d02330213d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:49:59.341282 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 762/1520] 10.222.0.64 () {58 vars in 1057 bytes} [Thu Jul 14 17:49:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:49:59.348891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9741a4-2b78-4995-93b4-af4369c34d05 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:49:59.362090 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9741a4-2b78-4995-93b4-af4369c34d05 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:49:59.367002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc7da0-f777-4aa0-9181-0e5a816c29e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:49:59.367646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc7da0-f777-4aa0-9181-0e5a816c29e5 service placement] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:49:59.367984 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc7da0-f777-4aa0-9181-0e5a816c29e5 service placement] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:49:59.368273 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc7da0-f777-4aa0-9181-0e5a816c29e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:49:59.377439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc7da0-f777-4aa0-9181-0e5a816c29e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:49:59.379007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc7da0-f777-4aa0-9181-0e5a816c29e5 service placement] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:49:59.389151 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 759/1521] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:49:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:50:00.349761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63379b76-9f5a-4424-9d25-450da293473a None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:00.350276 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63379b76-9f5a-4424-9d25-450da293473a None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:00.350276 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63379b76-9f5a-4424-9d25-450da293473a None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:00.351644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 17:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:50:00.357036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39863e9f-3bff-4f7e-b6a0-ab9b952deef5 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:00.357492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39863e9f-3bff-4f7e-b6a0-ab9b952deef5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:00.357675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39863e9f-3bff-4f7e-b6a0-ab9b952deef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:00.377702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-39863e9f-3bff-4f7e-b6a0-ab9b952deef5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:50:00.378934 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-39863e9f-3bff-4f7e-b6a0-ab9b952deef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:50:00.386838 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 17:50:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:50:00.543694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63379b76-9f5a-4424-9d25-450da293473a None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:50:00.558284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63379b76-9f5a-4424-9d25-450da293473a None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:50:00.563603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce9ae99-a928-476d-be70-2eecdcdf4f65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:50:00.564274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9ae99-a928-476d-be70-2eecdcdf4f65 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:00.564554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9ae99-a928-476d-be70-2eecdcdf4f65 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:00.564824 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce9ae99-a928-476d-be70-2eecdcdf4f65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:00.572886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9ae99-a928-476d-be70-2eecdcdf4f65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:50:00.574479 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce9ae99-a928-476d-be70-2eecdcdf4f65 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:50:00.584219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 764/1524] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:50:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:50:01.401702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953dddb-625f-4724-88b1-0073a689b868 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:01.402272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953dddb-625f-4724-88b1-0073a689b868 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:01.402272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953dddb-625f-4724-88b1-0073a689b868 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:01.403574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 17:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:50:01.408586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2a066-f19a-42db-a4d1-77a7c3d7a692 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:01.409030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2a066-f19a-42db-a4d1-77a7c3d7a692 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:01.409133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2a066-f19a-42db-a4d1-77a7c3d7a692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:01.427523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-55b2a066-f19a-42db-a4d1-77a7c3d7a692 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:50:01.428763 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-55b2a066-f19a-42db-a4d1-77a7c3d7a692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:50:01.437240 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 17:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:50:01.586876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953dddb-625f-4724-88b1-0073a689b868 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:50:01.601630 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953dddb-625f-4724-88b1-0073a689b868 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:50:01.607681 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9994c1d-fb95-4b10-93cd-f4743854f73d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:50:01.608354 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9994c1d-fb95-4b10-93cd-f4743854f73d service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:01.608636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9994c1d-fb95-4b10-93cd-f4743854f73d service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:01.608914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9994c1d-fb95-4b10-93cd-f4743854f73d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:01.617031 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9994c1d-fb95-4b10-93cd-f4743854f73d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:50:01.618638 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9994c1d-fb95-4b10-93cd-f4743854f73d service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:50:01.628051 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 762/1527] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:50:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:50:05.440583 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596ebc2-65a9-434a-9994-7d909d016012 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:05.441148 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596ebc2-65a9-434a-9994-7d909d016012 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:05.441148 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596ebc2-65a9-434a-9994-7d909d016012 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:05.442463 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:50:05.447833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613f424-4c58-4b84-beab-c9b3da9431c6 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:05.448324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613f424-4c58-4b84-beab-c9b3da9431c6 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:05.448429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613f424-4c58-4b84-beab-c9b3da9431c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:05.586535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-6613f424-4c58-4b84-beab-c9b3da9431c6 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:50:05.587761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6613f424-4c58-4b84-beab-c9b3da9431c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:50:05.595696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:50:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:50:05.672067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596ebc2-65a9-434a-9994-7d909d016012 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:50:05.674140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1596ebc2-65a9-434a-9994-7d909d016012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:50:05.702911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:50:05.732537 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596ebc2-65a9-434a-9994-7d909d016012 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:50:05.736041 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88301e1-0bd1-408d-ad4b-7bbed269092c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:50:05.736681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88301e1-0bd1-408d-ad4b-7bbed269092c service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:05.736962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88301e1-0bd1-408d-ad4b-7bbed269092c service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:05.737241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88301e1-0bd1-408d-ad4b-7bbed269092c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:05.746026 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88301e1-0bd1-408d-ad4b-7bbed269092c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:50:05.748095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88301e1-0bd1-408d-ad4b-7bbed269092c service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:50:05.757689 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 767/1530] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:50:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:50:07.821964 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6e7d9-500d-4a3e-9dc7-677bb43aeaa4 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:07.822480 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6e7d9-500d-4a3e-9dc7-677bb43aeaa4 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:07.822480 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6e7d9-500d-4a3e-9dc7-677bb43aeaa4 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:07.823903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:50:07.829317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa90dc-790d-42a0-8b25-1fb2e8e1b414 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:07.829822 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa90dc-790d-42a0-8b25-1fb2e8e1b414 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:07.829927 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa90dc-790d-42a0-8b25-1fb2e8e1b414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:07.847488 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-b6aa90dc-790d-42a0-8b25-1fb2e8e1b414 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:50:07.848731 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b6aa90dc-790d-42a0-8b25-1fb2e8e1b414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:50:07.856619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:50:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:50:07.894156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6e7d9-500d-4a3e-9dc7-677bb43aeaa4 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:50:07.904979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6e7d9-500d-4a3e-9dc7-677bb43aeaa4 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:50:07.908485 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69572441-d168-4f43-afda-617e3851a1a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:50:07.909107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69572441-d168-4f43-afda-617e3851a1a6 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:07.909392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69572441-d168-4f43-afda-617e3851a1a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:07.909675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69572441-d168-4f43-afda-617e3851a1a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:07.918513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69572441-d168-4f43-afda-617e3851a1a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:50:07.920470 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69572441-d168-4f43-afda-617e3851a1a6 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:50:07.930187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 765/1533] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:50:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:50:11.421697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36772b0b-6152-44fa-99de-e82379388374 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:11.422239 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36772b0b-6152-44fa-99de-e82379388374 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:11.422239 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36772b0b-6152-44fa-99de-e82379388374 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:11.423543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:50:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:50:11.429445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b2bfa-1991-4ee0-bf8b-0aeae21596af None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:11.429882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b2bfa-1991-4ee0-bf8b-0aeae21596af None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:11.430005 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096b2bfa-1991-4ee0-bf8b-0aeae21596af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:11.447547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-096b2bfa-1991-4ee0-bf8b-0aeae21596af None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:50:11.448779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-096b2bfa-1991-4ee0-bf8b-0aeae21596af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:50:11.456602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:50:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:50:11.760940 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36772b0b-6152-44fa-99de-e82379388374 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:50:11.762607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-36772b0b-6152-44fa-99de-e82379388374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:50:11.822594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36772b0b-6152-44fa-99de-e82379388374 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:50:11.826637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5f5b2-adfc-4c31-b221-514789fe7b9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:50:11.827290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5f5b2-adfc-4c31-b221-514789fe7b9c service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:11.827578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5f5b2-adfc-4c31-b221-514789fe7b9c service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:11.827860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5f5b2-adfc-4c31-b221-514789fe7b9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:11.837152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5f5b2-adfc-4c31-b221-514789fe7b9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:50:11.838588 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5f5b2-adfc-4c31-b221-514789fe7b9c service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:50:11.848696 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 770/1536] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:50:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:50:13.901096 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56291340-d411-41e7-9734-8f0522baeb9f None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:13.901699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56291340-d411-41e7-9734-8f0522baeb9f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:13.901699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56291340-d411-41e7-9734-8f0522baeb9f None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:13.903048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 17:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:50:13.908294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c51ec-e804-4e80-a552-6df8ba150624 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:13.908748 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c51ec-e804-4e80-a552-6df8ba150624 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:13.908845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c51ec-e804-4e80-a552-6df8ba150624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:13.927259 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-895c51ec-e804-4e80-a552-6df8ba150624 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:50:13.928498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-895c51ec-e804-4e80-a552-6df8ba150624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:50:13.936883 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 17:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:50:14.246701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56291340-d411-41e7-9734-8f0522baeb9f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:50:14.248494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-56291340-d411-41e7-9734-8f0522baeb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:50:14.261542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56291340-d411-41e7-9734-8f0522baeb9f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:50:14.265078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f9e1c-2a2c-4296-9805-3fea019340c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:50:14.265737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f9e1c-2a2c-4296-9805-3fea019340c5 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:50:14.266025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f9e1c-2a2c-4296-9805-3fea019340c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:50:14.266301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f9e1c-2a2c-4296-9805-3fea019340c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:50:14.275623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f9e1c-2a2c-4296-9805-3fea019340c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:50:14.277098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f9e1c-2a2c-4296-9805-3fea019340c5 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:50:14.287094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 768/1539] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:50:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:50:23.922345 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=47 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:50:24.282920 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=33 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:43.804863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c47d01-88c0-4a6d-831e-bce14fa10fc1 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:43.805928 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c47d01-88c0-4a6d-831e-bce14fa10fc1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:43.806436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c47d01-88c0-4a6d-831e-bce14fa10fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:43.833838 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-e6c47d01-88c0-4a6d-831e-bce14fa10fc1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:43.835450 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e6c47d01-88c0-4a6d-831e-bce14fa10fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:43.844560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 772/1540] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:43.854037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f48813-58fc-47f7-88ba-d906a48c277e None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:43.854602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f48813-58fc-47f7-88ba-d906a48c277e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:43.854602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f48813-58fc-47f7-88ba-d906a48c277e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:43.880995 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-90f48813-58fc-47f7-88ba-d906a48c277e None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:43.882241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-90f48813-58fc-47f7-88ba-d906a48c277e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:43.897073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 769/1541] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:43.902147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c47d01-88c0-4a6d-831e-bce14fa10fc1 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:43.913517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c47d01-88c0-4a6d-831e-bce14fa10fc1 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:43.925830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdbd39-f044-4033-bcf2-9ff9d8f90dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:43.927467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdbd39-f044-4033-bcf2-9ff9d8f90dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:43.928129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdbd39-f044-4033-bcf2-9ff9d8f90dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:43.928648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bdbd39-f044-4033-bcf2-9ff9d8f90dd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:43.929698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdbd39-f044-4033-bcf2-9ff9d8f90dd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:51:43.930274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdbd39-f044-4033-bcf2-9ff9d8f90dd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:43.932331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bdbd39-f044-4033-bcf2-9ff9d8f90dd5 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:43.955656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 773/1542] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:51:43.964484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f48813-58fc-47f7-88ba-d906a48c277e None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:43.981720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f48813-58fc-47f7-88ba-d906a48c277e None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:43.993361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d0162-358f-4e4c-923c-c258f0a4754d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:43.994063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d0162-358f-4e4c-923c-c258f0a4754d admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:43.994596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d0162-358f-4e4c-923c-c258f0a4754d admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:43.994887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d0162-358f-4e4c-923c-c258f0a4754d admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:43.995591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d0162-358f-4e4c-923c-c258f0a4754d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:44.001269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d0162-358f-4e4c-923c-c258f0a4754d admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:44.049771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 770/1543] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:43 2022] POST /identity/v3/projects => generated 399 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:44.054648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdbd39-f044-4033-bcf2-9ff9d8f90dd5 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:44.069636 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bdbd39-f044-4033-bcf2-9ff9d8f90dd5 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:44.298611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5393e0-0f13-48cf-97e2-45655c4cfae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:44.299906 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5393e0-0f13-48cf-97e2-45655c4cfae8 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.300224 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5393e0-0f13-48cf-97e2-45655c4cfae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.300506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5393e0-0f13-48cf-97e2-45655c4cfae8 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.301222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5393e0-0f13-48cf-97e2-45655c4cfae8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:44.303446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5393e0-0f13-48cf-97e2-45655c4cfae8 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:44.336578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 774/1544] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/users => generated 300 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:44.340976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d0162-358f-4e4c-923c-c258f0a4754d admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:44.362697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d0162-358f-4e4c-923c-c258f0a4754d admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:44.370327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b4e56-631a-4ccb-b665-8d39da233352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:44.371501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b4e56-631a-4ccb-b665-8d39da233352 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.372033 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b4e56-631a-4ccb-b665-8d39da233352 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.372544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b4e56-631a-4ccb-b665-8d39da233352 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.373502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b4e56-631a-4ccb-b665-8d39da233352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:44.375336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b4e56-631a-4ccb-b665-8d39da233352 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:44.390717 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 771/1545] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:44.394627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5393e0-0f13-48cf-97e2-45655c4cfae8 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:44.406010 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5393e0-0f13-48cf-97e2-45655c4cfae8 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:44.416420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e5defd-00ef-4b86-91a9-bc226d3106bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:44.417230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5defd-00ef-4b86-91a9-bc226d3106bd admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.417526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5defd-00ef-4b86-91a9-bc226d3106bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.417807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e5defd-00ef-4b86-91a9-bc226d3106bd admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.418622 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5defd-00ef-4b86-91a9-bc226d3106bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:44.420282 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e5defd-00ef-4b86-91a9-bc226d3106bd admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:44.436477 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b4e56-631a-4ccb-b665-8d39da233352 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:44.436706 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 775/1546] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:44.437967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962726e-d382-4920-90a3-2b32b625e001 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.438263 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962726e-d382-4920-90a3-2b32b625e001 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.438544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962726e-d382-4920-90a3-2b32b625e001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.455881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b4e56-631a-4ccb-b665-8d39da233352 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:44.468905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-3962726e-d382-4920-90a3-2b32b625e001 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:44.470253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3962726e-d382-4920-90a3-2b32b625e001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:44.476804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a3966-fba4-493a-9602-a0cdced6d0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:44.478182 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a3966-fba4-493a-9602-a0cdced6d0a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.478754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a3966-fba4-493a-9602-a0cdced6d0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.479255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a3966-fba4-493a-9602-a0cdced6d0a7 admin admin] PATH_INFO: `/v3/projects/b7ee3a419a5c4b8d81d103e92a77c4bf/users/2fff9718251f49fb8b532b312fb17074/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.485860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 776/1547] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:44.486467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38d98a-ab46-4a53-9884-0b891a68f10d None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.486755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38d98a-ab46-4a53-9884-0b891a68f10d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.487044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38d98a-ab46-4a53-9884-0b891a68f10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.503618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a3966-fba4-493a-9602-a0cdced6d0a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7ee3a419a5c4b8d81d103e92a77c4bf, user_id=2fff9718251f49fb8b532b312fb17074, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:44.510850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3a3966-fba4-493a-9602-a0cdced6d0a7 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:44.518339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-ff38d98a-ab46-4a53-9884-0b891a68f10d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:44.518339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ff38d98a-ab46-4a53-9884-0b891a68f10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:44.531278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 777/1548] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:44.532758 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297586-f46b-4f69-ae09-b13f37694598 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.533052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297586-f46b-4f69-ae09-b13f37694598 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.533333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297586-f46b-4f69-ae09-b13f37694598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.535292 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 772/1549] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:44 2022] PUT /identity/v3/projects/b7ee3a419a5c4b8d81d103e92a77c4bf/users/2fff9718251f49fb8b532b312fb17074/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:51:44.540415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fecbc62-0bf3-439c-8865-67abc0767944 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.541000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fecbc62-0bf3-439c-8865-67abc0767944 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.541502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fecbc62-0bf3-439c-8865-67abc0767944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.569792 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-ee297586-f46b-4f69-ae09-b13f37694598 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:44.570647 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ee297586-f46b-4f69-ae09-b13f37694598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:44.576647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-7fecbc62-0bf3-439c-8865-67abc0767944 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:44.578170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7fecbc62-0bf3-439c-8865-67abc0767944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:44.658575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 778/1550] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:44.661743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b1bc9-cbc1-471a-8001-38e3207b3527 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.662027 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b1bc9-cbc1-471a-8001-38e3207b3527 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.662303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b1bc9-cbc1-471a-8001-38e3207b3527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.691426 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 773/1551] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:51:44.693833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55369d23-8e72-47c5-8ae3-0259c93ff7c5 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.694294 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55369d23-8e72-47c5-8ae3-0259c93ff7c5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.697509 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55369d23-8e72-47c5-8ae3-0259c93ff7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.698775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-fa0b1bc9-cbc1-471a-8001-38e3207b3527 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:44.700454 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-fa0b1bc9-cbc1-471a-8001-38e3207b3527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:44.713875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 779/1552] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:44.713875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5244862-967b-4466-ba23-8b6221377f03 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.713875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5244862-967b-4466-ba23-8b6221377f03 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.713875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5244862-967b-4466-ba23-8b6221377f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.716133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-55369d23-8e72-47c5-8ae3-0259c93ff7c5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:44.717345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-55369d23-8e72-47c5-8ae3-0259c93ff7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:44.727632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 774/1553] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:44.732205 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55369d23-8e72-47c5-8ae3-0259c93ff7c5 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:44.737937 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-f5244862-967b-4466-ba23-8b6221377f03 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:44.738131 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55369d23-8e72-47c5-8ae3-0259c93ff7c5 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:44.740177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f5244862-967b-4466-ba23-8b6221377f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:44.744030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66d366-c584-42b6-af60-f96a23d08b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:44.744698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66d366-c584-42b6-af60-f96a23d08b46 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.744982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66d366-c584-42b6-af60-f96a23d08b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.745276 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66d366-c584-42b6-af60-f96a23d08b46 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.745991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66d366-c584-42b6-af60-f96a23d08b46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:51:44.746302 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66d366-c584-42b6-af60-f96a23d08b46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:44.747317 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: WARNING py.warnings [None req-7e66d366-c584-42b6-af60-f96a23d08b46 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:51:44.747317 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: warnings.warn(msg) Jul 14 17:51:44.747317 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: Jul 14 17:51:44.748218 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66d366-c584-42b6-af60-f96a23d08b46 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:44.750179 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 780/1554] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:44.751445 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97682e84-66e9-459a-b349-a3fe92dc5305 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.751728 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97682e84-66e9-459a-b349-a3fe92dc5305 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.752027 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97682e84-66e9-459a-b349-a3fe92dc5305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.765840 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 775/1555] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:51:44.767439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2546df9-77fc-4484-bf2c-ce922d3fc230 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.767740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2546df9-77fc-4484-bf2c-ce922d3fc230 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.768059 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2546df9-77fc-4484-bf2c-ce922d3fc230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.770014 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-97682e84-66e9-459a-b349-a3fe92dc5305 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:44.771233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-97682e84-66e9-459a-b349-a3fe92dc5305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:44.779341 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 781/1556] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:44.782684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97682e84-66e9-459a-b349-a3fe92dc5305 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:44.786344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-a2546df9-77fc-4484-bf2c-ce922d3fc230 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:44.787545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a2546df9-77fc-4484-bf2c-ce922d3fc230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:44.795503 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 776/1557] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:44.796146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2546df9-77fc-4484-bf2c-ce922d3fc230 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:44.806627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2546df9-77fc-4484-bf2c-ce922d3fc230 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:44.811587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03beb59-00b3-46f6-90c7-b60a63ea3fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:44.812288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03beb59-00b3-46f6-90c7-b60a63ea3fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.812574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03beb59-00b3-46f6-90c7-b60a63ea3fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.812854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03beb59-00b3-46f6-90c7-b60a63ea3fc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.813687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03beb59-00b3-46f6-90c7-b60a63ea3fc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:51:44.813926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03beb59-00b3-46f6-90c7-b60a63ea3fc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:44.815344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:44.815839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03beb59-00b3-46f6-90c7-b60a63ea3fc1 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:44.835265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 777/1558] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:51:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:51:44.835265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604e5d8-94f1-4c2c-9a4e-eac1d423c4cd None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.835265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604e5d8-94f1-4c2c-9a4e-eac1d423c4cd None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.835265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604e5d8-94f1-4c2c-9a4e-eac1d423c4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.845203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97682e84-66e9-459a-b349-a3fe92dc5305 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:44.847720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-0604e5d8-94f1-4c2c-9a4e-eac1d423c4cd None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:44.848869 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd125e-f056-4829-b119-87c5bbb72852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:44.849047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0604e5d8-94f1-4c2c-9a4e-eac1d423c4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:44.849496 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd125e-f056-4829-b119-87c5bbb72852 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.849779 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd125e-f056-4829-b119-87c5bbb72852 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.850055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd125e-f056-4829-b119-87c5bbb72852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.857077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 778/1559] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:44.859033 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b324a0-b456-4004-8efe-cfe24b923c1f None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.859511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b324a0-b456-4004-8efe-cfe24b923c1f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.859648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd125e-f056-4829-b119-87c5bbb72852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:44.859836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b324a0-b456-4004-8efe-cfe24b923c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.861109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd125e-f056-4829-b119-87c5bbb72852 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:44.871193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 782/1560] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:44.872461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30d929-f5ca-425a-8697-f9a49b9473a3 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.872852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30d929-f5ca-425a-8697-f9a49b9473a3 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.873133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30d929-f5ca-425a-8697-f9a49b9473a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.878166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-55b324a0-b456-4004-8efe-cfe24b923c1f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:44.879362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-55b324a0-b456-4004-8efe-cfe24b923c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:44.888388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 779/1561] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:44.889261 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b324a0-b456-4004-8efe-cfe24b923c1f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:44.899043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-6a30d929-f5ca-425a-8697-f9a49b9473a3 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:44.900303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b324a0-b456-4004-8efe-cfe24b923c1f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:44.900434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6a30d929-f5ca-425a-8697-f9a49b9473a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:44.904981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f42238-7033-48dc-b79e-377fe1a1d1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:44.905661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42238-7033-48dc-b79e-377fe1a1d1f4 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.905954 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42238-7033-48dc-b79e-377fe1a1d1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.906228 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f42238-7033-48dc-b79e-377fe1a1d1f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.906920 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f42238-7033-48dc-b79e-377fe1a1d1f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:44.908852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 783/1562] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:44.910079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dff186-876e-4fc4-b32b-e4c141523a10 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.910408 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dff186-876e-4fc4-b32b-e4c141523a10 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.910692 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dff186-876e-4fc4-b32b-e4c141523a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.913026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f42238-7033-48dc-b79e-377fe1a1d1f4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:44.935429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-f4dff186-876e-4fc4-b32b-e4c141523a10 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:44.936665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f4dff186-876e-4fc4-b32b-e4c141523a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:44.944779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 780/1563] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:44.945411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 784/1564] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:44.946221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dff186-876e-4fc4-b32b-e4c141523a10 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:44.946944 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb968c-1aac-40d2-94e9-3bd2aeda194a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:44.947393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb968c-1aac-40d2-94e9-3bd2aeda194a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:44.948126 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb968c-1aac-40d2-94e9-3bd2aeda194a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:44.956359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dff186-876e-4fc4-b32b-e4c141523a10 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:44.971453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-49eb968c-1aac-40d2-94e9-3bd2aeda194a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:44.972873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-49eb968c-1aac-40d2-94e9-3bd2aeda194a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:44.995367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:45.023091 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92928d-20f4-4657-b06a-4ec213708c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.023849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92928d-20f4-4657-b06a-4ec213708c29 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.024179 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92928d-20f4-4657-b06a-4ec213708c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.024562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92928d-20f4-4657-b06a-4ec213708c29 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.025259 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92928d-20f4-4657-b06a-4ec213708c29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.026625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: WARNING py.warnings [None req-5e92928d-20f4-4657-b06a-4ec213708c29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:51:45.026625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: warnings.warn(msg) Jul 14 17:51:45.026625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: Jul 14 17:51:45.027467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92928d-20f4-4657-b06a-4ec213708c29 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.058732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 785/1565] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:45.062753 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 781/1566] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:45.062880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92928d-20f4-4657-b06a-4ec213708c29 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.062978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf4a19-d54d-4438-9892-6222aeebfc56 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.062978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf4a19-d54d-4438-9892-6222aeebfc56 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.062978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bf4a19-d54d-4438-9892-6222aeebfc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.090517 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-50bf4a19-d54d-4438-9892-6222aeebfc56 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:45.092105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-50bf4a19-d54d-4438-9892-6222aeebfc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:45.143762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92928d-20f4-4657-b06a-4ec213708c29 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.148597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4538-278e-494f-b0b4-20bc5b4cc118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.149279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4538-278e-494f-b0b4-20bc5b4cc118 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.149565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4538-278e-494f-b0b4-20bc5b4cc118 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.149834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4538-278e-494f-b0b4-20bc5b4cc118 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.150575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb4538-278e-494f-b0b4-20bc5b4cc118 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:51:45.150884 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb4538-278e-494f-b0b4-20bc5b4cc118 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.152532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb4538-278e-494f-b0b4-20bc5b4cc118 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.158236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 782/1567] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:45.159279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf4a19-d54d-4438-9892-6222aeebfc56 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.170758 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 786/1568] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:51:45.171093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bf4a19-d54d-4438-9892-6222aeebfc56 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.172684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4538-278e-494f-b0b4-20bc5b4cc118 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.176673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edfada-20b8-42fa-bdc5-eae238e5f63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.178651 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edfada-20b8-42fa-bdc5-eae238e5f63a admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.179013 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edfada-20b8-42fa-bdc5-eae238e5f63a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.179383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edfada-20b8-42fa-bdc5-eae238e5f63a admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.180441 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edfada-20b8-42fa-bdc5-eae238e5f63a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:51:45.180624 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edfada-20b8-42fa-bdc5-eae238e5f63a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.182580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edfada-20b8-42fa-bdc5-eae238e5f63a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.187655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4538-278e-494f-b0b4-20bc5b4cc118 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.193336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafd96e-8860-49e2-b8c3-0f020ab18e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.194976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafd96e-8860-49e2-b8c3-0f020ab18e9f admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.195373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafd96e-8860-49e2-b8c3-0f020ab18e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.195726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafd96e-8860-49e2-b8c3-0f020ab18e9f admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.196835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cafd96e-8860-49e2-b8c3-0f020ab18e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.205652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 783/1569] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:51:45.205854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cafd96e-8860-49e2-b8c3-0f020ab18e9f admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.207166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edfada-20b8-42fa-bdc5-eae238e5f63a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.218878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edfada-20b8-42fa-bdc5-eae238e5f63a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.224966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd532bdd-6590-461a-b029-62fbf28f4b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.225646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd532bdd-6590-461a-b029-62fbf28f4b28 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.225935 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd532bdd-6590-461a-b029-62fbf28f4b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.226219 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd532bdd-6590-461a-b029-62fbf28f4b28 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.227102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd532bdd-6590-461a-b029-62fbf28f4b28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:51:45.227427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd532bdd-6590-461a-b029-62fbf28f4b28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.229063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd532bdd-6590-461a-b029-62fbf28f4b28 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.230069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 787/1570] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:51:45 2022] POST /identity/v3/users => generated 293 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:45.230833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafd96e-8860-49e2-b8c3-0f020ab18e9f admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.243674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafd96e-8860-49e2-b8c3-0f020ab18e9f admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.246263 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 784/1571] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:51:45.247112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd532bdd-6590-461a-b029-62fbf28f4b28 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.257866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65048c4a-f724-4ae2-8466-f9ebf82cd8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.258563 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65048c4a-f724-4ae2-8466-f9ebf82cd8c1 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.258851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65048c4a-f724-4ae2-8466-f9ebf82cd8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.259131 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65048c4a-f724-4ae2-8466-f9ebf82cd8c1 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.259249 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd532bdd-6590-461a-b029-62fbf28f4b28 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.259888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65048c4a-f724-4ae2-8466-f9ebf82cd8c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.261839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65048c4a-f724-4ae2-8466-f9ebf82cd8c1 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.266235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e06d53-9a08-4f1f-b652-a48e151ec90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.266235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e06d53-9a08-4f1f-b652-a48e151ec90a admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.266235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e06d53-9a08-4f1f-b652-a48e151ec90a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.266641 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e06d53-9a08-4f1f-b652-a48e151ec90a admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.267169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e06d53-9a08-4f1f-b652-a48e151ec90a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:51:45.267510 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e06d53-9a08-4f1f-b652-a48e151ec90a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.269237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e06d53-9a08-4f1f-b652-a48e151ec90a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.279602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:45.283798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 785/1572] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:51:45.284961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e06d53-9a08-4f1f-b652-a48e151ec90a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.288047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 788/1573] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:51:45 2022] POST /identity/v3/users => generated 290 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:45.288047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65048c4a-f724-4ae2-8466-f9ebf82cd8c1 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.301746 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e06d53-9a08-4f1f-b652-a48e151ec90a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.303381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65048c4a-f724-4ae2-8466-f9ebf82cd8c1 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.306489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944b9e6-fcee-4b15-8522-23106d490a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.307167 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944b9e6-fcee-4b15-8522-23106d490a91 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.307456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944b9e6-fcee-4b15-8522-23106d490a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.307760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5944b9e6-fcee-4b15-8522-23106d490a91 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.308519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944b9e6-fcee-4b15-8522-23106d490a91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:51:45.308833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944b9e6-fcee-4b15-8522-23106d490a91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.310479 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e17e7-3700-456f-b128-998c91526de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.310698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5944b9e6-fcee-4b15-8522-23106d490a91 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.311497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e17e7-3700-456f-b128-998c91526de1 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.311972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e17e7-3700-456f-b128-998c91526de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.312588 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e17e7-3700-456f-b128-998c91526de1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.313941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e17e7-3700-456f-b128-998c91526de1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.316048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e17e7-3700-456f-b128-998c91526de1 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.324057 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 786/1574] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:51:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:51:45.325212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944b9e6-fcee-4b15-8522-23106d490a91 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.337901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5944b9e6-fcee-4b15-8522-23106d490a91 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.343063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d98f64-d6da-4235-b277-7526f1dd5991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.343725 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d98f64-d6da-4235-b277-7526f1dd5991 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.344180 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d98f64-d6da-4235-b277-7526f1dd5991 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.344461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d98f64-d6da-4235-b277-7526f1dd5991 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.345161 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d98f64-d6da-4235-b277-7526f1dd5991 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.347103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d98f64-d6da-4235-b277-7526f1dd5991 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.357112 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 789/1575] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:45 2022] POST /identity/v3/projects => generated 391 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:45.360299 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e17e7-3700-456f-b128-998c91526de1 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.383800 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 787/1576] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:45 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:45.388579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d98f64-d6da-4235-b277-7526f1dd5991 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.440166 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e17e7-3700-456f-b128-998c91526de1 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.455506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:45.471423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d98f64-d6da-4235-b277-7526f1dd5991 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.477031 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207f67b-b055-433c-bbfd-042c42411a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.477721 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207f67b-b055-433c-bbfd-042c42411a8a admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.478007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207f67b-b055-433c-bbfd-042c42411a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.478291 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7207f67b-b055-433c-bbfd-042c42411a8a admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.478997 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207f67b-b055-433c-bbfd-042c42411a8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.480995 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7207f67b-b055-433c-bbfd-042c42411a8a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.509957 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9bff1-4412-4330-aab4-96f658e76cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.510741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9bff1-4412-4330-aab4-96f658e76cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.511089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9bff1-4412-4330-aab4-96f658e76cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.511435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9bff1-4412-4330-aab4-96f658e76cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.512279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd9bff1-4412-4330-aab4-96f658e76cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.516771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 788/1577] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:45 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:45.516931 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd9bff1-4412-4330-aab4-96f658e76cd4 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.517037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207f67b-b055-433c-bbfd-042c42411a8a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.530183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 790/1578] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:45.531212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9bff1-4412-4330-aab4-96f658e76cd4 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.542906 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9bff1-4412-4330-aab4-96f658e76cd4 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.606906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7207f67b-b055-433c-bbfd-042c42411a8a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.610610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178afcf-29de-4dbb-9f38-c9902b55e978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.611590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178afcf-29de-4dbb-9f38-c9902b55e978 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.611713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa675d7-fb6b-4884-97b4-f8db600ef723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.612115 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178afcf-29de-4dbb-9f38-c9902b55e978 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.612419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa675d7-fb6b-4884-97b4-f8db600ef723 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.612575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0178afcf-29de-4dbb-9f38-c9902b55e978 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.612703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa675d7-fb6b-4884-97b4-f8db600ef723 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.613022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa675d7-fb6b-4884-97b4-f8db600ef723 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.613617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178afcf-29de-4dbb-9f38-c9902b55e978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.613742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa675d7-fb6b-4884-97b4-f8db600ef723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.615411 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa675d7-fb6b-4884-97b4-f8db600ef723 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.615896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0178afcf-29de-4dbb-9f38-c9902b55e978 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.629890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 789/1579] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:45.629890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa675d7-fb6b-4884-97b4-f8db600ef723 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.638258 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa675d7-fb6b-4884-97b4-f8db600ef723 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.643537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85babe3b-e607-459e-a29e-24a27ff6ba1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.644295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85babe3b-e607-459e-a29e-24a27ff6ba1c admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.644582 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85babe3b-e607-459e-a29e-24a27ff6ba1c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.644862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85babe3b-e607-459e-a29e-24a27ff6ba1c admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.646112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85babe3b-e607-459e-a29e-24a27ff6ba1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.648109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85babe3b-e607-459e-a29e-24a27ff6ba1c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.655267 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 791/1580] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:45 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:45.656287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178afcf-29de-4dbb-9f38-c9902b55e978 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.668999 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:45.683332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 790/1581] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:45 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:45.685118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85babe3b-e607-459e-a29e-24a27ff6ba1c admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.738104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0178afcf-29de-4dbb-9f38-c9902b55e978 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.781665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85babe3b-e607-459e-a29e-24a27ff6ba1c admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.788403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b54151-f003-4f2f-b5e1-994bbde88901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.789447 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b54151-f003-4f2f-b5e1-994bbde88901 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.789803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b54151-f003-4f2f-b5e1-994bbde88901 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.790175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b54151-f003-4f2f-b5e1-994bbde88901 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.790949 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b54151-f003-4f2f-b5e1-994bbde88901 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.792149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: WARNING py.warnings [None req-e2b54151-f003-4f2f-b5e1-994bbde88901 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:51:45.792149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: warnings.warn(msg) Jul 14 17:51:45.792149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: Jul 14 17:51:45.793526 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b54151-f003-4f2f-b5e1-994bbde88901 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.811080 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fa115-e656-4d2f-b4f3-f431a449c5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.811940 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fa115-e656-4d2f-b4f3-f431a449c5b2 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.812307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fa115-e656-4d2f-b4f3-f431a449c5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.812685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7fa115-e656-4d2f-b4f3-f431a449c5b2 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.813567 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fa115-e656-4d2f-b4f3-f431a449c5b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.815502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7fa115-e656-4d2f-b4f3-f431a449c5b2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.821069 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 791/1582] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:51:45 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:45.821069 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b54151-f003-4f2f-b5e1-994bbde88901 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.831488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b54151-f003-4f2f-b5e1-994bbde88901 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.844317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 792/1583] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:51:45 2022] POST /identity/v3/users => generated 296 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:45.844549 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d9df4-9a4d-4a11-9328-0d81a97d4df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.844549 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d9df4-9a4d-4a11-9328-0d81a97d4df6 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.844549 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d9df4-9a4d-4a11-9328-0d81a97d4df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.844814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fa115-e656-4d2f-b4f3-f431a449c5b2 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.844898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d9df4-9a4d-4a11-9328-0d81a97d4df6 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.844898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d9df4-9a4d-4a11-9328-0d81a97d4df6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.845805 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d9df4-9a4d-4a11-9328-0d81a97d4df6 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.855258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7fa115-e656-4d2f-b4f3-f431a449c5b2 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.865101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca85fce-d5f2-42e0-9960-37e1054834ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.866114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca85fce-d5f2-42e0-9960-37e1054834ba admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.866618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca85fce-d5f2-42e0-9960-37e1054834ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.867643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 792/1584] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:51:45 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:45.871290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca85fce-d5f2-42e0-9960-37e1054834ba admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.871290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca85fce-d5f2-42e0-9960-37e1054834ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.871501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d9df4-9a4d-4a11-9328-0d81a97d4df6 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.871604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca85fce-d5f2-42e0-9960-37e1054834ba admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.879592 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d9df4-9a4d-4a11-9328-0d81a97d4df6 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.883761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 793/1585] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:45.884620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca85fce-d5f2-42e0-9960-37e1054834ba admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.889358 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325a638-feab-482b-be34-76f47fdafa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.890112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325a638-feab-482b-be34-76f47fdafa98 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.890399 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325a638-feab-482b-be34-76f47fdafa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.890676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d325a638-feab-482b-be34-76f47fdafa98 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.894002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325a638-feab-482b-be34-76f47fdafa98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.896454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d325a638-feab-482b-be34-76f47fdafa98 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.897677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca85fce-d5f2-42e0-9960-37e1054834ba admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.902492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe81b61-2f69-4292-9e61-ca192395e0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.903185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe81b61-2f69-4292-9e61-ca192395e0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.903442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe81b61-2f69-4292-9e61-ca192395e0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.903722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe81b61-2f69-4292-9e61-ca192395e0e2 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.904564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe81b61-2f69-4292-9e61-ca192395e0e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.906437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe81b61-2f69-4292-9e61-ca192395e0e2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.907759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 793/1586] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:45.908600 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325a638-feab-482b-be34-76f47fdafa98 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.921668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d325a638-feab-482b-be34-76f47fdafa98 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.927000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a44d4-9d3b-4919-91ce-5fee31c3206d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.931521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a44d4-9d3b-4919-91ce-5fee31c3206d admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.931676 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 794/1587] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:51:45 2022] POST /identity/v3/users => generated 284 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:45.931774 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a44d4-9d3b-4919-91ce-5fee31c3206d admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.931774 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4a44d4-9d3b-4919-91ce-5fee31c3206d admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.931774 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a44d4-9d3b-4919-91ce-5fee31c3206d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.932196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfee2b-ac38-4a94-a064-b14ead90bd5a None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.932196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfee2b-ac38-4a94-a064-b14ead90bd5a None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.932196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdfee2b-ac38-4a94-a064-b14ead90bd5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.932495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4a44d4-9d3b-4919-91ce-5fee31c3206d admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.940069 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:45.961650 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4cdfee2b-ac38-4a94-a064-b14ead90bd5a None None] MFA Rules not processed for user `2fff9718251f49fb8b532b312fb17074`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:45.961849 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 794/1588] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:51:45 2022] POST /identity/v3/users => generated 288 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:45.962574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a44d4-9d3b-4919-91ce-5fee31c3206d admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:45.963554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4cdfee2b-ac38-4a94-a064-b14ead90bd5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:45.972533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4a44d4-9d3b-4919-91ce-5fee31c3206d admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:45.977448 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd9719-a6cd-49f6-821a-89c22636f7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:45.978118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd9719-a6cd-49f6-821a-89c22636f7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:45.978409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd9719-a6cd-49f6-821a-89c22636f7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:45.978694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd9719-a6cd-49f6-821a-89c22636f7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:45.979428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd9719-a6cd-49f6-821a-89c22636f7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:45.981065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd9719-a6cd-49f6-821a-89c22636f7e4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:45.992387 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 795/1589] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:45.993265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd9719-a6cd-49f6-821a-89c22636f7e4 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.003061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd9719-a6cd-49f6-821a-89c22636f7e4 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.008622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d13ec9-d371-4437-976a-6d4edca4cec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.009283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d13ec9-d371-4437-976a-6d4edca4cec7 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.009587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d13ec9-d371-4437-976a-6d4edca4cec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.009853 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d13ec9-d371-4437-976a-6d4edca4cec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.010583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d13ec9-d371-4437-976a-6d4edca4cec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.012204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d13ec9-d371-4437-976a-6d4edca4cec7 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.024436 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 796/1590] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:46.025332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d13ec9-d371-4437-976a-6d4edca4cec7 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.034825 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d13ec9-d371-4437-976a-6d4edca4cec7 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.037362 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 795/1591] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:46.038235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfee2b-ac38-4a94-a064-b14ead90bd5a None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.042241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbbf2f-52a9-49c2-a91c-e533d4e5842b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.042974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbbf2f-52a9-49c2-a91c-e533d4e5842b admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.043342 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbbf2f-52a9-49c2-a91c-e533d4e5842b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.043690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbbf2f-52a9-49c2-a91c-e533d4e5842b admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.044535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbbf2f-52a9-49c2-a91c-e533d4e5842b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.044983 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:46.046162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbbf2f-52a9-49c2-a91c-e533d4e5842b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.049793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdfee2b-ac38-4a94-a064-b14ead90bd5a None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.056769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418acbd-804a-4e43-bd4c-9c3cb1fd47e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.057799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418acbd-804a-4e43-bd4c-9c3cb1fd47e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.057926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 797/1592] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:46.058048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418acbd-804a-4e43-bd4c-9c3cb1fd47e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.058246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418acbd-804a-4e43-bd4c-9c3cb1fd47e6 admin admin] PATH_INFO: `/v3/projects/69d2b6a5a4534cadb34e626ce33238df/users/c4178d34dc114f46af725c9dc362d224/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.058594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbbf2f-52a9-49c2-a91c-e533d4e5842b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.070021 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbbf2f-52a9-49c2-a91c-e533d4e5842b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.075640 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b57fb-89d3-458e-9989-7b9513c48e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.076556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b57fb-89d3-458e-9989-7b9513c48e75 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.076736 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418acbd-804a-4e43-bd4c-9c3cb1fd47e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d2b6a5a4534cadb34e626ce33238df, user_id=c4178d34dc114f46af725c9dc362d224, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.076864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b57fb-89d3-458e-9989-7b9513c48e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.077105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4b57fb-89d3-458e-9989-7b9513c48e75 admin admin] PATH_INFO: `/v3/projects/65ba80b0874640d289c54ad4ac5a1018/users/be1201ac0aa7484f96e271410342a6b9/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.077760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: WARNING py.warnings [None req-c418acbd-804a-4e43-bd4c-9c3cb1fd47e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:51:46.077760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: warnings.warn(msg) Jul 14 17:51:46.077760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: Jul 14 17:51:46.079706 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418acbd-804a-4e43-bd4c-9c3cb1fd47e6 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.093299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b57fb-89d3-458e-9989-7b9513c48e75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65ba80b0874640d289c54ad4ac5a1018, user_id=be1201ac0aa7484f96e271410342a6b9, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.100847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4b57fb-89d3-458e-9989-7b9513c48e75 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.107367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 796/1593] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:46 2022] PUT /identity/v3/projects/69d2b6a5a4534cadb34e626ce33238df/users/c4178d34dc114f46af725c9dc362d224/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:46.107367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418acbd-804a-4e43-bd4c-9c3cb1fd47e6 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.114195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 798/1594] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:46 2022] PUT /identity/v3/projects/65ba80b0874640d289c54ad4ac5a1018/users/be1201ac0aa7484f96e271410342a6b9/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:46.115720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b57fb-89d3-458e-9989-7b9513c48e75 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.175767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418acbd-804a-4e43-bd4c-9c3cb1fd47e6 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.182020 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6ae4a-00bb-4ed5-80b0-a54f3b5c81a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.182748 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6ae4a-00bb-4ed5-80b0-a54f3b5c81a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.183038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6ae4a-00bb-4ed5-80b0-a54f3b5c81a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.183485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d6ae4a-00bb-4ed5-80b0-a54f3b5c81a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.183632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4b57fb-89d3-458e-9989-7b9513c48e75 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.184526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6ae4a-00bb-4ed5-80b0-a54f3b5c81a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.186138 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d6ae4a-00bb-4ed5-80b0-a54f3b5c81a9 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.188548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99436546-0db6-4d7a-be94-9e71311773b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.189240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99436546-0db6-4d7a-be94-9e71311773b5 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.189522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99436546-0db6-4d7a-be94-9e71311773b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.189805 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99436546-0db6-4d7a-be94-9e71311773b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.190530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99436546-0db6-4d7a-be94-9e71311773b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.192171 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99436546-0db6-4d7a-be94-9e71311773b5 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.197234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 797/1595] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:46.198106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6ae4a-00bb-4ed5-80b0-a54f3b5c81a9 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.203792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 799/1596] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:46.204773 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99436546-0db6-4d7a-be94-9e71311773b5 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.209900 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d6ae4a-00bb-4ed5-80b0-a54f3b5c81a9 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.214475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5a425-4d9f-41cc-95a9-9321de1736b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.214631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99436546-0db6-4d7a-be94-9e71311773b5 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.215067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5a425-4d9f-41cc-95a9-9321de1736b1 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.215346 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5a425-4d9f-41cc-95a9-9321de1736b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.215619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db5a425-4d9f-41cc-95a9-9321de1736b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.216370 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5a425-4d9f-41cc-95a9-9321de1736b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.217940 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db5a425-4d9f-41cc-95a9-9321de1736b1 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.219110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e73f1-c86f-41e1-a716-1f340ea9032f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.219757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e73f1-c86f-41e1-a716-1f340ea9032f admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.220072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e73f1-c86f-41e1-a716-1f340ea9032f admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.220353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e73f1-c86f-41e1-a716-1f340ea9032f admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.221101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e73f1-c86f-41e1-a716-1f340ea9032f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.222662 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e73f1-c86f-41e1-a716-1f340ea9032f admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.228105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 798/1597] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:46.228873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5a425-4d9f-41cc-95a9-9321de1736b1 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.233490 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 800/1598] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:46.234687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e73f1-c86f-41e1-a716-1f340ea9032f admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.238554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db5a425-4d9f-41cc-95a9-9321de1736b1 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.243678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b62557-798b-4319-943b-b84dbc545393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.244370 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b62557-798b-4319-943b-b84dbc545393 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.244646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b62557-798b-4319-943b-b84dbc545393 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.244916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b62557-798b-4319-943b-b84dbc545393 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.245607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b62557-798b-4319-943b-b84dbc545393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.245938 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e73f1-c86f-41e1-a716-1f340ea9032f admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.247481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b62557-798b-4319-943b-b84dbc545393 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.250604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616f898-268a-4114-b25c-cea8e7f21b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.251292 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f898-268a-4114-b25c-cea8e7f21b8c admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.251581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f898-268a-4114-b25c-cea8e7f21b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.251863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616f898-268a-4114-b25c-cea8e7f21b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.252666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616f898-268a-4114-b25c-cea8e7f21b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.254269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616f898-268a-4114-b25c-cea8e7f21b8c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.266229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 801/1599] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:46.267551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f551af-8891-4b49-84f5-d4068b508055 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.267833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f551af-8891-4b49-84f5-d4068b508055 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.268142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f551af-8891-4b49-84f5-d4068b508055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.273942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 799/1600] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:46 2022] POST /identity/v3/projects => generated 405 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:46.275407 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc36573-78fc-4560-989c-d4f72549b9a5 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.275707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc36573-78fc-4560-989c-d4f72549b9a5 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.276021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc36573-78fc-4560-989c-d4f72549b9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.284598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-52f551af-8891-4b49-84f5-d4068b508055 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:46.285823 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-52f551af-8891-4b49-84f5-d4068b508055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:46.292606 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-3dc36573-78fc-4560-989c-d4f72549b9a5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:46.293784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3dc36573-78fc-4560-989c-d4f72549b9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:46.347441 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 802/1601] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:46.348583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f551af-8891-4b49-84f5-d4068b508055 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.352506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 800/1602] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:46.354061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc36573-78fc-4560-989c-d4f72549b9a5 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.359441 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f551af-8891-4b49-84f5-d4068b508055 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.364688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29f43e-e1fc-4672-9103-f1b9c9bbb224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.365475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29f43e-e1fc-4672-9103-f1b9c9bbb224 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.365726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc36573-78fc-4560-989c-d4f72549b9a5 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.365840 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29f43e-e1fc-4672-9103-f1b9c9bbb224 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.366033 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29f43e-e1fc-4672-9103-f1b9c9bbb224 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.366766 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29f43e-e1fc-4672-9103-f1b9c9bbb224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.368404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29f43e-e1fc-4672-9103-f1b9c9bbb224 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.370429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f847bc-ceec-4b2f-8ae4-da774bac456b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.370913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:46.371094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f847bc-ceec-4b2f-8ae4-da774bac456b admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.371742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f847bc-ceec-4b2f-8ae4-da774bac456b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.372067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f847bc-ceec-4b2f-8ae4-da774bac456b admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.372811 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f847bc-ceec-4b2f-8ae4-da774bac456b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.374388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f847bc-ceec-4b2f-8ae4-da774bac456b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.382100 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 803/1603] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:46.383188 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29f43e-e1fc-4672-9103-f1b9c9bbb224 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.385321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 801/1604] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:46.386107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f847bc-ceec-4b2f-8ae4-da774bac456b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.393904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29f43e-e1fc-4672-9103-f1b9c9bbb224 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.395520 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f847bc-ceec-4b2f-8ae4-da774bac456b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.398496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a77af4-5f28-4ea9-878b-7a9e8d4a62ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.399376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a77af4-5f28-4ea9-878b-7a9e8d4a62ef admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.399660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a77af4-5f28-4ea9-878b-7a9e8d4a62ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.399959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a77af4-5f28-4ea9-878b-7a9e8d4a62ef admin admin] PATH_INFO: `/v3/projects/ceae870c225d45cca6d22ed0f5b072fe/users/d0d3f805c35448a194bba74d75ba2d14/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.400303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec80b53-36eb-4eb0-8ce6-4aee7634b753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.401172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec80b53-36eb-4eb0-8ce6-4aee7634b753 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.401455 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec80b53-36eb-4eb0-8ce6-4aee7634b753 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.401733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec80b53-36eb-4eb0-8ce6-4aee7634b753 admin admin] PATH_INFO: `/v3/projects/ab58d5c4c0194b58ad47dd5e4494389f/users/52c794c11ab74f9ca9bad75857043a62/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.416139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a77af4-5f28-4ea9-878b-7a9e8d4a62ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceae870c225d45cca6d22ed0f5b072fe, user_id=d0d3f805c35448a194bba74d75ba2d14, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.416867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec80b53-36eb-4eb0-8ce6-4aee7634b753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab58d5c4c0194b58ad47dd5e4494389f, user_id=52c794c11ab74f9ca9bad75857043a62, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.418966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a77af4-5f28-4ea9-878b-7a9e8d4a62ef admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.419488 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec80b53-36eb-4eb0-8ce6-4aee7634b753 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.434105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 804/1605] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:46 2022] PUT /identity/v3/projects/ceae870c225d45cca6d22ed0f5b072fe/users/d0d3f805c35448a194bba74d75ba2d14/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:46.435145 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a77af4-5f28-4ea9-878b-7a9e8d4a62ef admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.436933 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 802/1606] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:46 2022] PUT /identity/v3/projects/ab58d5c4c0194b58ad47dd5e4494389f/users/52c794c11ab74f9ca9bad75857043a62/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:46.437767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec80b53-36eb-4eb0-8ce6-4aee7634b753 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.451229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:46.499107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a77af4-5f28-4ea9-878b-7a9e8d4a62ef admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.503733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8ace6-d745-4fed-8ba5-3bac3ca0c7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.504661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8ace6-d745-4fed-8ba5-3bac3ca0c7ed admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.504968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8ace6-d745-4fed-8ba5-3bac3ca0c7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.505239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8ace6-d745-4fed-8ba5-3bac3ca0c7ed admin admin] PATH_INFO: `/v3/projects/8c5795bd83a04c00a3d5efff38435557/users/c94fd85ceb8840e7af970e3191357caa/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.507880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec80b53-36eb-4eb0-8ce6-4aee7634b753 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.512459 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc69279-9c4c-45fb-8158-f6ca45f05380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.513134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc69279-9c4c-45fb-8158-f6ca45f05380 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.513419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc69279-9c4c-45fb-8158-f6ca45f05380 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.513691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc69279-9c4c-45fb-8158-f6ca45f05380 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.514378 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc69279-9c4c-45fb-8158-f6ca45f05380 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.516363 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc69279-9c4c-45fb-8158-f6ca45f05380 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.519662 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8ace6-d745-4fed-8ba5-3bac3ca0c7ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c5795bd83a04c00a3d5efff38435557, user_id=c94fd85ceb8840e7af970e3191357caa, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.522328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8ace6-d745-4fed-8ba5-3bac3ca0c7ed admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.540792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 805/1607] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:46 2022] PUT /identity/v3/projects/8c5795bd83a04c00a3d5efff38435557/users/c94fd85ceb8840e7af970e3191357caa/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:46.540792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8ace6-d745-4fed-8ba5-3bac3ca0c7ed admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.541073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 803/1608] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:51:46 2022] POST /identity/v3/users => generated 309 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:46.541073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc69279-9c4c-45fb-8158-f6ca45f05380 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.541274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c6e8ace6-d745-4fed-8ba5-3bac3ca0c7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:46.597974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc69279-9c4c-45fb-8158-f6ca45f05380 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.598471 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8ace6-d745-4fed-8ba5-3bac3ca0c7ed admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.602173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90413322-3415-4380-a029-81b862924b0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.602575 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789235cb-4f74-4909-bf99-1e5495f23dbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.602837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90413322-3415-4380-a029-81b862924b0c service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.603122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90413322-3415-4380-a029-81b862924b0c service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.603222 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789235cb-4f74-4909-bf99-1e5495f23dbc service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.603402 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90413322-3415-4380-a029-81b862924b0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.603498 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789235cb-4f74-4909-bf99-1e5495f23dbc service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.603764 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789235cb-4f74-4909-bf99-1e5495f23dbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.664511 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90413322-3415-4380-a029-81b862924b0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.664756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789235cb-4f74-4909-bf99-1e5495f23dbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.666027 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90413322-3415-4380-a029-81b862924b0c service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.666154 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789235cb-4f74-4909-bf99-1e5495f23dbc service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.675960 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 806/1609] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:46.676128 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 804/1610] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:46.676608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789235cb-4f74-4909-bf99-1e5495f23dbc service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.676933 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90413322-3415-4380-a029-81b862924b0c service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.687693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789235cb-4f74-4909-bf99-1e5495f23dbc service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.688328 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90413322-3415-4380-a029-81b862924b0c service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.692724 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b26e7d6-b20d-4890-aa49-8066986843da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.693706 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26e7d6-b20d-4890-aa49-8066986843da admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.694179 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26e7d6-b20d-4890-aa49-8066986843da admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.694537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b26e7d6-b20d-4890-aa49-8066986843da admin admin] PATH_INFO: `/v3/projects/b021b9c607ff4edfb338294a1faee71f/users/197bd8804cd44905b65bbbc5cb191bfa/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.695891 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad252d-0e6b-4bd9-b103-b8f8a9232534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.697023 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad252d-0e6b-4bd9-b103-b8f8a9232534 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.697432 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:46.698480 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad252d-0e6b-4bd9-b103-b8f8a9232534 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.698925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad252d-0e6b-4bd9-b103-b8f8a9232534 admin admin] PATH_INFO: `/v3/projects/e51408d5653d460f82009b65e238281f/users/84c38f6ff5af4ba79c80fb581baacb0b/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.717918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad252d-0e6b-4bd9-b103-b8f8a9232534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e51408d5653d460f82009b65e238281f, user_id=84c38f6ff5af4ba79c80fb581baacb0b, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.720808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad252d-0e6b-4bd9-b103-b8f8a9232534 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.726612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26e7d6-b20d-4890-aa49-8066986843da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b021b9c607ff4edfb338294a1faee71f, user_id=197bd8804cd44905b65bbbc5cb191bfa, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.730236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b26e7d6-b20d-4890-aa49-8066986843da admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.742217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 805/1611] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:46 2022] PUT /identity/v3/projects/e51408d5653d460f82009b65e238281f/users/84c38f6ff5af4ba79c80fb581baacb0b/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:46.746317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae2ec6-e264-48b1-9d49-3687b78ea521 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.746600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae2ec6-e264-48b1-9d49-3687b78ea521 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.746870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ae2ec6-e264-48b1-9d49-3687b78ea521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.753514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 807/1612] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:46 2022] PUT /identity/v3/projects/b021b9c607ff4edfb338294a1faee71f/users/197bd8804cd44905b65bbbc5cb191bfa/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:46.754849 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37ee84-ce0e-4ccc-b030-f8b44466e177 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.755176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37ee84-ce0e-4ccc-b030-f8b44466e177 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.755439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec37ee84-ce0e-4ccc-b030-f8b44466e177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.782974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-22ae2ec6-e264-48b1-9d49-3687b78ea521 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:46.784834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-22ae2ec6-e264-48b1-9d49-3687b78ea521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:46.788923 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-ec37ee84-ce0e-4ccc-b030-f8b44466e177 None None] MFA Rules not processed for user `d0d3f805c35448a194bba74d75ba2d14`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:46.790409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ec37ee84-ce0e-4ccc-b030-f8b44466e177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:46.812009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:46.867189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 806/1613] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:46.868043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae2ec6-e264-48b1-9d49-3687b78ea521 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:46.878514 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ae2ec6-e264-48b1-9d49-3687b78ea521 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:46.884790 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9fd958-c904-4b8e-bf08-ec984315c9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:46.885481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fd958-c904-4b8e-bf08-ec984315c9d6 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.885768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fd958-c904-4b8e-bf08-ec984315c9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.886132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fd958-c904-4b8e-bf08-ec984315c9d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.886891 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fd958-c904-4b8e-bf08-ec984315c9d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:46.888681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9fd958-c904-4b8e-bf08-ec984315c9d6 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:46.901155 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 807/1614] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:46.905964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144798a-a28f-49c6-9d93-5f2a1b143459 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.906260 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144798a-a28f-49c6-9d93-5f2a1b143459 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.906539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1144798a-a28f-49c6-9d93-5f2a1b143459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.912432 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 808/1615] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:46.914531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e88156-8ab5-4199-81ee-2449634d7544 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.915884 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e88156-8ab5-4199-81ee-2449634d7544 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.916377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e88156-8ab5-4199-81ee-2449634d7544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:46.944981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-64e88156-8ab5-4199-81ee-2449634d7544 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:46.946464 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-1144798a-a28f-49c6-9d93-5f2a1b143459 None None] MFA Rules not processed for user `c94fd85ceb8840e7af970e3191357caa`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:46.947699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1144798a-a28f-49c6-9d93-5f2a1b143459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:46.949370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-64e88156-8ab5-4199-81ee-2449634d7544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:46.962097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 809/1616] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:46.967036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcaa9b-53b3-44ae-b3f6-388f29b9f561 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:46.967240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcaa9b-53b3-44ae-b3f6-388f29b9f561 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:46.967713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bcaa9b-53b3-44ae-b3f6-388f29b9f561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:47.003902 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-22bcaa9b-53b3-44ae-b3f6-388f29b9f561 None None] MFA Rules not processed for user `197bd8804cd44905b65bbbc5cb191bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:47.005200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-22bcaa9b-53b3-44ae-b3f6-388f29b9f561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:47.022697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 808/1617] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:47.023751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144798a-a28f-49c6-9d93-5f2a1b143459 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:47.109304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 810/1618] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:47.111771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcaa9b-53b3-44ae-b3f6-388f29b9f561 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:47.121377 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1144798a-a28f-49c6-9d93-5f2a1b143459 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:47.127275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bcaa9b-53b3-44ae-b3f6-388f29b9f561 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:47.129091 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6a76a-c5cb-42ae-82d6-6db861ced6c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:47.129764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6a76a-c5cb-42ae-82d6-6db861ced6c7 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:47.130147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6a76a-c5cb-42ae-82d6-6db861ced6c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:47.130359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6a76a-c5cb-42ae-82d6-6db861ced6c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:47.133939 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd4d51-d43f-4a7d-b069-5b333bbaba12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:47.135729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd4d51-d43f-4a7d-b069-5b333bbaba12 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:47.136050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd4d51-d43f-4a7d-b069-5b333bbaba12 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:47.136431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd4d51-d43f-4a7d-b069-5b333bbaba12 admin admin] PATH_INFO: `/v3/projects/b5852053939d420c8ee0d372826597b7/users/05fe506a87a443dab657b8a0a48f9f6a/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:47.150335 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6a76a-c5cb-42ae-82d6-6db861ced6c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:47.151801 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6a76a-c5cb-42ae-82d6-6db861ced6c7 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:47.158310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd4d51-d43f-4a7d-b069-5b333bbaba12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5852053939d420c8ee0d372826597b7, user_id=05fe506a87a443dab657b8a0a48f9f6a, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:47.162806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcd4d51-d43f-4a7d-b069-5b333bbaba12 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:47.182005 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 809/1619] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:47.183427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d665e5-872f-49f0-9331-26affe1c261b None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:47.183821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d665e5-872f-49f0-9331-26affe1c261b None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:47.183971 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d665e5-872f-49f0-9331-26affe1c261b None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:47.185798 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 810/1620] 10.222.0.64 () {54 vars in 961 bytes} [Thu Jul 14 17:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:51:47.186544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d665e5-872f-49f0-9331-26affe1c261b None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:47.198356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 811/1621] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:47 2022] PUT /identity/v3/projects/b5852053939d420c8ee0d372826597b7/users/05fe506a87a443dab657b8a0a48f9f6a/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:47.200315 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd4d51-d43f-4a7d-b069-5b333bbaba12 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:47.256821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:47.290752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:47.299099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d665e5-872f-49f0-9331-26affe1c261b None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:47.304913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c3230-fdc0-4cb0-9479-0df0e69e0201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:47.305598 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c3230-fdc0-4cb0-9479-0df0e69e0201 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:47.305890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c3230-fdc0-4cb0-9479-0df0e69e0201 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:47.306176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c3230-fdc0-4cb0-9479-0df0e69e0201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:47.312377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcd4d51-d43f-4a7d-b069-5b333bbaba12 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:47.320910 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c3230-fdc0-4cb0-9479-0df0e69e0201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:47.321785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2d30f-c0f1-440e-b50a-f4cb91226090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:47.322566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c3230-fdc0-4cb0-9479-0df0e69e0201 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:47.322942 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2d30f-c0f1-440e-b50a-f4cb91226090 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:47.323544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2d30f-c0f1-440e-b50a-f4cb91226090 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:47.324108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2d30f-c0f1-440e-b50a-f4cb91226090 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:47.325122 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2d30f-c0f1-440e-b50a-f4cb91226090 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:47.327447 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2d30f-c0f1-440e-b50a-f4cb91226090 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:47.337447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 811/1622] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:47.338233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c3230-fdc0-4cb0-9479-0df0e69e0201 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:47.370220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 812/1623] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:47 2022] POST /identity/v3/projects => generated 387 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:47.372540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cc3a6-8d1b-4d7e-b4c1-73ce465b5497 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:47.373047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cc3a6-8d1b-4d7e-b4c1-73ce465b5497 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:47.373489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4cc3a6-8d1b-4d7e-b4c1-73ce465b5497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:47.412714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c3230-fdc0-4cb0-9479-0df0e69e0201 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:47.421898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-8c4cc3a6-8d1b-4d7e-b4c1-73ce465b5497 None None] MFA Rules not processed for user `6423f54152064b909d03c0908d636993`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:47.423398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8c4cc3a6-8d1b-4d7e-b4c1-73ce465b5497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:47.477046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1242c-7cc8-4de5-9dea-97dbf3b9b002 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:47.477993 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1242c-7cc8-4de5-9dea-97dbf3b9b002 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:47.478537 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1242c-7cc8-4de5-9dea-97dbf3b9b002 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:47.479170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b1242c-7cc8-4de5-9dea-97dbf3b9b002 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:47.537990 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 813/1624] 10.222.0.64 () {58 vars in 1058 bytes} [Thu Jul 14 17:51:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:47.539268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cc3a6-8d1b-4d7e-b4c1-73ce465b5497 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:47.575986 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1242c-7cc8-4de5-9dea-97dbf3b9b002 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:47.579841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b1242c-7cc8-4de5-9dea-97dbf3b9b002 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:47.601618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 812/1625] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:47 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:47.603388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1242c-7cc8-4de5-9dea-97dbf3b9b002 service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:47.674760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4cc3a6-8d1b-4d7e-b4c1-73ce465b5497 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:47.689037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c476ef-0d11-41a2-82d4-b62023a60138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:47.691584 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c476ef-0d11-41a2-82d4-b62023a60138 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:47.693932 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c476ef-0d11-41a2-82d4-b62023a60138 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:47.694356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c476ef-0d11-41a2-82d4-b62023a60138 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:47.695649 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c476ef-0d11-41a2-82d4-b62023a60138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:47.701798 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c476ef-0d11-41a2-82d4-b62023a60138 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:47.726072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 814/1626] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:47.727173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c476ef-0d11-41a2-82d4-b62023a60138 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:47.752207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c476ef-0d11-41a2-82d4-b62023a60138 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:47.756508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b1242c-7cc8-4de5-9dea-97dbf3b9b002 service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:47.760643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23e0e7-2d1a-4393-ae9e-d82305c03e5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:47.761571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23e0e7-2d1a-4393-ae9e-d82305c03e5a service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:47.761936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23e0e7-2d1a-4393-ae9e-d82305c03e5a service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:47.762596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f23e0e7-2d1a-4393-ae9e-d82305c03e5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:47.764772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b7ba7-0fb8-42dc-b7ea-3e789e9258fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:47.765649 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b7ba7-0fb8-42dc-b7ea-3e789e9258fb admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:47.765943 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b7ba7-0fb8-42dc-b7ea-3e789e9258fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:47.766226 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b7ba7-0fb8-42dc-b7ea-3e789e9258fb admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:47.766938 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b7ba7-0fb8-42dc-b7ea-3e789e9258fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:47.769216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966b7ba7-0fb8-42dc-b7ea-3e789e9258fb admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:47.787478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:47.811974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 813/1627] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:51:47 2022] POST /identity/v3/users => generated 300 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:47.811974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b7ba7-0fb8-42dc-b7ea-3e789e9258fb admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:47.823660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:47.831871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b7ba7-0fb8-42dc-b7ea-3e789e9258fb admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:47.840338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87a389-751a-4c82-aae5-01b8031b78d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:47.841649 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87a389-751a-4c82-aae5-01b8031b78d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:47.842171 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87a389-751a-4c82-aae5-01b8031b78d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:47.842656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e87a389-751a-4c82-aae5-01b8031b78d8 admin admin] PATH_INFO: `/v3/projects/b5852053939d420c8ee0d372826597b7/users/05fe506a87a443dab657b8a0a48f9f6a/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:47.845774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87a389-751a-4c82-aae5-01b8031b78d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5852053939d420c8ee0d372826597b7, user_id=05fe506a87a443dab657b8a0a48f9f6a, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:47.853161 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e87a389-751a-4c82-aae5-01b8031b78d8 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:47.901259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23e0e7-2d1a-4393-ae9e-d82305c03e5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:47.901510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 814/1628] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:47 2022] PUT /identity/v3/projects/b5852053939d420c8ee0d372826597b7/users/05fe506a87a443dab657b8a0a48f9f6a/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:47.903253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87a389-751a-4c82-aae5-01b8031b78d8 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:47.904546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f23e0e7-2d1a-4393-ae9e-d82305c03e5a service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:47.941216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 815/1629] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:48.027221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e87a389-751a-4c82-aae5-01b8031b78d8 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:48.039388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b445b-9525-4fdc-9a14-001ecbc55cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:48.039587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b445b-9525-4fdc-9a14-001ecbc55cce admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:48.039837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b445b-9525-4fdc-9a14-001ecbc55cce admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:48.040266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b445b-9525-4fdc-9a14-001ecbc55cce admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:48.041021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b445b-9525-4fdc-9a14-001ecbc55cce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:48.056705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b445b-9525-4fdc-9a14-001ecbc55cce admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:48.088303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 815/1630] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:48.093722 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23e0e7-2d1a-4393-ae9e-d82305c03e5a service nova] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:48.113668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f23e0e7-2d1a-4393-ae9e-d82305c03e5a service nova] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:48.126822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b52b06-3b8f-4741-8c9f-2f2d39a8d493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:48.128649 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b52b06-3b8f-4741-8c9f-2f2d39a8d493 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:48.129078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b52b06-3b8f-4741-8c9f-2f2d39a8d493 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:48.129397 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b52b06-3b8f-4741-8c9f-2f2d39a8d493 admin admin] PATH_INFO: `/v3/projects/1515b50247e84ea9ba5a42cf4cfdf9c2/users/b265ecb516d54a8a9940cb86cb4fff2b/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:48.166330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b52b06-3b8f-4741-8c9f-2f2d39a8d493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1515b50247e84ea9ba5a42cf4cfdf9c2, user_id=b265ecb516d54a8a9940cb86cb4fff2b, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:48.169614 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b52b06-3b8f-4741-8c9f-2f2d39a8d493 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:48.194775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 816/1631] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:48 2022] PUT /identity/v3/projects/1515b50247e84ea9ba5a42cf4cfdf9c2/users/b265ecb516d54a8a9940cb86cb4fff2b/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:48.198965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b445b-9525-4fdc-9a14-001ecbc55cce admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:48.215209 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b445b-9525-4fdc-9a14-001ecbc55cce admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:48.389726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeccdba-1fd0-4014-b4d4-c556714ebac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:48.390739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeccdba-1fd0-4014-b4d4-c556714ebac2 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:48.390984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeccdba-1fd0-4014-b4d4-c556714ebac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:48.391332 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeccdba-1fd0-4014-b4d4-c556714ebac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:48.394233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeccdba-1fd0-4014-b4d4-c556714ebac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:48.397171 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeccdba-1fd0-4014-b4d4-c556714ebac2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:48.435137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 816/1632] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:48.436731 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b52b06-3b8f-4741-8c9f-2f2d39a8d493 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:48.450840 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b52b06-3b8f-4741-8c9f-2f2d39a8d493 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:48.462648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fd7ff-9fa7-47dc-a96b-d6f4e56f9889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:48.463574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fd7ff-9fa7-47dc-a96b-d6f4e56f9889 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:48.463859 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fd7ff-9fa7-47dc-a96b-d6f4e56f9889 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:48.464629 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fd7ff-9fa7-47dc-a96b-d6f4e56f9889 admin admin] PATH_INFO: `/v3/projects/1515b50247e84ea9ba5a42cf4cfdf9c2/users/b265ecb516d54a8a9940cb86cb4fff2b/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:48.473501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fd7ff-9fa7-47dc-a96b-d6f4e56f9889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1515b50247e84ea9ba5a42cf4cfdf9c2, user_id=b265ecb516d54a8a9940cb86cb4fff2b, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:48.476915 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fd7ff-9fa7-47dc-a96b-d6f4e56f9889 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:48.500631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 817/1633] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:48 2022] PUT /identity/v3/projects/1515b50247e84ea9ba5a42cf4cfdf9c2/users/b265ecb516d54a8a9940cb86cb4fff2b/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:48.518110 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871553a-4759-4dc1-adbc-aab5cafc8558 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:48.518525 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871553a-4759-4dc1-adbc-aab5cafc8558 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:48.518703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9871553a-4759-4dc1-adbc-aab5cafc8558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:48.568720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39846555-257f-4747-a0d8-f9e5965deff9 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:48.568720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39846555-257f-4747-a0d8-f9e5965deff9 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:48.569005 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39846555-257f-4747-a0d8-f9e5965deff9 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:48.570369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 818/1634] 10.222.0.64 () {54 vars in 912 bytes} [Thu Jul 14 17:51:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:51:48.577402 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae4a9d-ad6a-4402-8c94-613edc19d03e None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:48.577690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae4a9d-ad6a-4402-8c94-613edc19d03e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:48.577978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae4a9d-ad6a-4402-8c94-613edc19d03e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:48.580791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-9871553a-4759-4dc1-adbc-aab5cafc8558 None None] MFA Rules not processed for user `b265ecb516d54a8a9940cb86cb4fff2b`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:48.582016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9871553a-4759-4dc1-adbc-aab5cafc8558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:48.622120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-adae4a9d-ad6a-4402-8c94-613edc19d03e None None] MFA Rules not processed for user `6423f54152064b909d03c0908d636993`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:48.623356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-adae4a9d-ad6a-4402-8c94-613edc19d03e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:48.744512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:48.768934 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 819/1635] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 17:51:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:48.772693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 817/1636] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:48 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:48.790421 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae4a9d-ad6a-4402-8c94-613edc19d03e None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:48.850699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871553a-4759-4dc1-adbc-aab5cafc8558 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:48.901908 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae4a9d-ad6a-4402-8c94-613edc19d03e None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:48.906108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8399a-e100-4316-8c07-5cad9c0e0ff3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:48.906731 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8399a-e100-4316-8c07-5cad9c0e0ff3 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:48.907251 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8399a-e100-4316-8c07-5cad9c0e0ff3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:48.907541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8399a-e100-4316-8c07-5cad9c0e0ff3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:48.919804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8399a-e100-4316-8c07-5cad9c0e0ff3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:48.921990 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba8399a-e100-4316-8c07-5cad9c0e0ff3 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:48.941151 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 820/1637] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:48.978237 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9871553a-4759-4dc1-adbc-aab5cafc8558 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:48.983133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0bc79-801e-45a4-bde4-b96101bf00f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:48.984002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0bc79-801e-45a4-bde4-b96101bf00f8 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:48.984472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0bc79-801e-45a4-bde4-b96101bf00f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:48.984758 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0bc79-801e-45a4-bde4-b96101bf00f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:49.011283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0bc79-801e-45a4-bde4-b96101bf00f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:49.014576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd0bc79-801e-45a4-bde4-b96101bf00f8 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:49.037713 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 818/1638] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:49.054023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8399a-e100-4316-8c07-5cad9c0e0ff3 service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:49.092312 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:49.151143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8399a-e100-4316-8c07-5cad9c0e0ff3 service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:49.156210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a1821-09a7-4e1a-8385-b8ba17308454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:49.159351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a1821-09a7-4e1a-8385-b8ba17308454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:49.159652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a1821-09a7-4e1a-8385-b8ba17308454 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:49.159945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a1821-09a7-4e1a-8385-b8ba17308454 admin admin] PATH_INFO: `/v3/users/c94fd85ceb8840e7af970e3191357caa` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:49.163513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a1821-09a7-4e1a-8385-b8ba17308454 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c94fd85ceb8840e7af970e3191357caa)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:49.165578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a1821-09a7-4e1a-8385-b8ba17308454 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:49.234777 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e29a1821-09a7-4e1a-8385-b8ba17308454 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c94fd85ceb8840e7af970e3191357caa'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:49.254966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e29a1821-09a7-4e1a-8385-b8ba17308454 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c94fd85ceb8840e7af970e3191357caa'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:49.268385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e29a1821-09a7-4e1a-8385-b8ba17308454 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c94fd85ceb8840e7af970e3191357caa'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:49.269391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e29a1821-09a7-4e1a-8385-b8ba17308454 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c94fd85ceb8840e7af970e3191357caa'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:49.280186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e29a1821-09a7-4e1a-8385-b8ba17308454 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c94fd85ceb8840e7af970e3191357caa'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:49.292864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 821/1639] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:51:49 2022] DELETE /identity/v3/users/c94fd85ceb8840e7af970e3191357caa => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:49.295134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d585da-87d8-42dd-8aa4-4e878a478478 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:49.295429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d585da-87d8-42dd-8aa4-4e878a478478 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:49.295709 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d585da-87d8-42dd-8aa4-4e878a478478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:49.317841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-d0d585da-87d8-42dd-8aa4-4e878a478478 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:49.319089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d0d585da-87d8-42dd-8aa4-4e878a478478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:49.393828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 819/1640] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:49.431315 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a1821-09a7-4e1a-8385-b8ba17308454 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:49.433779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e29a1821-09a7-4e1a-8385-b8ba17308454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:49.515989 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a1821-09a7-4e1a-8385-b8ba17308454 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:49.520164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956b3256-97e7-4eb5-a1ff-e4f54099d1b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:49.522141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b3256-97e7-4eb5-a1ff-e4f54099d1b7 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:49.522428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b3256-97e7-4eb5-a1ff-e4f54099d1b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:49.522703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956b3256-97e7-4eb5-a1ff-e4f54099d1b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:49.534636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b3256-97e7-4eb5-a1ff-e4f54099d1b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:49.536332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956b3256-97e7-4eb5-a1ff-e4f54099d1b7 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:49.552052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 822/1641] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:50.321428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba3d90-6386-4a72-9459-cda3a9da4072 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:50.322063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba3d90-6386-4a72-9459-cda3a9da4072 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:50.322334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba3d90-6386-4a72-9459-cda3a9da4072 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:50.324226 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 820/1642] 10.222.0.64 () {54 vars in 961 bytes} [Thu Jul 14 17:51:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:51:50.333056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447372f9-cde7-4fd5-b9ca-035d01a50f79 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:50.334424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447372f9-cde7-4fd5-b9ca-035d01a50f79 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:50.334424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447372f9-cde7-4fd5-b9ca-035d01a50f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:50.375739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-447372f9-cde7-4fd5-b9ca-035d01a50f79 None None] MFA Rules not processed for user `6423f54152064b909d03c0908d636993`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:50.377676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-447372f9-cde7-4fd5-b9ca-035d01a50f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:50.509295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:50.579256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 823/1643] 10.222.0.64 () {58 vars in 1058 bytes} [Thu Jul 14 17:51:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:50.585470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a170ac5-7f31-4ae4-ac1e-a69e4189ec13 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:50.586133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a170ac5-7f31-4ae4-ac1e-a69e4189ec13 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:50.586606 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a170ac5-7f31-4ae4-ac1e-a69e4189ec13 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:50.588657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 821/1644] 10.222.0.64 () {54 vars in 912 bytes} [Thu Jul 14 17:51:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:51:50.594681 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8455048-c06d-4b15-9cf9-5a2769f869e2 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:50.595309 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8455048-c06d-4b15-9cf9-5a2769f869e2 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:50.596103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8455048-c06d-4b15-9cf9-5a2769f869e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:50.637771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b8455048-c06d-4b15-9cf9-5a2769f869e2 None None] MFA Rules not processed for user `6423f54152064b909d03c0908d636993`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:50.639196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b8455048-c06d-4b15-9cf9-5a2769f869e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:50.673951 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a170ac5-7f31-4ae4-ac1e-a69e4189ec13 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:50.676284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0a170ac5-7f31-4ae4-ac1e-a69e4189ec13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:50.678702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 824/1645] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 17:51:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:51:50.759402 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a170ac5-7f31-4ae4-ac1e-a69e4189ec13 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:50.764492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b48e7-eca7-4b0a-af59-a45167cac766 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:50.765906 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b48e7-eca7-4b0a-af59-a45167cac766 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:50.766518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b48e7-eca7-4b0a-af59-a45167cac766 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:50.767031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b48e7-eca7-4b0a-af59-a45167cac766 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:50.780608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b48e7-eca7-4b0a-af59-a45167cac766 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:50.782488 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b48e7-eca7-4b0a-af59-a45167cac766 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:50.799484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 822/1646] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:50.814054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c076e-8b8b-4b7a-8ded-84b9bf25d007 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:50.814349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c076e-8b8b-4b7a-8ded-84b9bf25d007 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:50.814637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c076e-8b8b-4b7a-8ded-84b9bf25d007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:50.822802 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b48e7-eca7-4b0a-af59-a45167cac766 service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:50.829719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-648b48e7-eca7-4b0a-af59-a45167cac766 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:50.885444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b48e7-eca7-4b0a-af59-a45167cac766 service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:50.901259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-5e6c076e-8b8b-4b7a-8ded-84b9bf25d007 None None] MFA Rules not processed for user `05fe506a87a443dab657b8a0a48f9f6a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:50.902388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5e6c076e-8b8b-4b7a-8ded-84b9bf25d007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:50.906631 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:50.909535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:50.909535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:50.909535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] PATH_INFO: `/v3/projects/8c5795bd83a04c00a3d5efff38435557` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:50.909833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c5795bd83a04c00a3d5efff38435557)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:50.911599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:51.001611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:51.040510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c5795bd83a04c00a3d5efff38435557', 'request_id': 'req-547a113f-ec01-4efc-a275-5646879996b3'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:51.105401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c5795bd83a04c00a3d5efff38435557', 'request_id': 'req-547a113f-ec01-4efc-a275-5646879996b3'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:51.105401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c5795bd83a04c00a3d5efff38435557', 'request_id': 'req-547a113f-ec01-4efc-a275-5646879996b3'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:51.105401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] The token cache is being invalidate because project 8c5795bd83a04c00a3d5efff38435557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:51:51.105401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:51.113680 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:51.120527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 823/1647] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:51:50 2022] DELETE /identity/v3/projects/8c5795bd83a04c00a3d5efff38435557 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:51.124230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:51.129333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:51.155260 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 825/1648] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:50 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:51.363645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547a113f-ec01-4efc-a275-5646879996b3 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:51.382185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0ec3d1-71f0-468c-bec6-1e130df43784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:51.383468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ec3d1-71f0-468c-bec6-1e130df43784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:51.383902 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ec3d1-71f0-468c-bec6-1e130df43784 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:51.384278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0ec3d1-71f0-468c-bec6-1e130df43784 admin admin] PATH_INFO: `/v3/users/b265ecb516d54a8a9940cb86cb4fff2b` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:51.386212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ec3d1-71f0-468c-bec6-1e130df43784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b265ecb516d54a8a9940cb86cb4fff2b)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:51.387638 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: WARNING py.warnings [None req-cb0ec3d1-71f0-468c-bec6-1e130df43784 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:51:51.387638 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: warnings.warn(msg) Jul 14 17:51:51.387638 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: Jul 14 17:51:51.390919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0ec3d1-71f0-468c-bec6-1e130df43784 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:51.515073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-cb0ec3d1-71f0-468c-bec6-1e130df43784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b265ecb516d54a8a9940cb86cb4fff2b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:51.562386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-cb0ec3d1-71f0-468c-bec6-1e130df43784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b265ecb516d54a8a9940cb86cb4fff2b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:51.563453 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-cb0ec3d1-71f0-468c-bec6-1e130df43784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b265ecb516d54a8a9940cb86cb4fff2b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:51.584587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-cb0ec3d1-71f0-468c-bec6-1e130df43784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b265ecb516d54a8a9940cb86cb4fff2b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:51.585926 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-cb0ec3d1-71f0-468c-bec6-1e130df43784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b265ecb516d54a8a9940cb86cb4fff2b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:51.630114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 824/1649] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:51:51 2022] DELETE /identity/v3/users/b265ecb516d54a8a9940cb86cb4fff2b => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:52.481183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c076e-8b8b-4b7a-8ded-84b9bf25d007 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:52.482710 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5e6c076e-8b8b-4b7a-8ded-84b9bf25d007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:52.504048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c076e-8b8b-4b7a-8ded-84b9bf25d007 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:52.621695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3271e3-f709-4055-8585-f40d797564d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:52.623678 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3271e3-f709-4055-8585-f40d797564d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:52.624438 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3271e3-f709-4055-8585-f40d797564d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:52.624994 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3271e3-f709-4055-8585-f40d797564d5 admin admin] PATH_INFO: `/v3/users/d0d3f805c35448a194bba74d75ba2d14` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:52.627439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3271e3-f709-4055-8585-f40d797564d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0d3f805c35448a194bba74d75ba2d14)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:52.633047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3271e3-f709-4055-8585-f40d797564d5 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:52.665998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e797f3-4342-4a14-9e40-78378904e597 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:52.666832 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e797f3-4342-4a14-9e40-78378904e597 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:52.667263 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e797f3-4342-4a14-9e40-78378904e597 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:52.668907 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 825/1650] 10.222.0.64 () {54 vars in 921 bytes} [Thu Jul 14 17:51:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:51:52.719548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5a3271e3-f709-4055-8585-f40d797564d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0d3f805c35448a194bba74d75ba2d14'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:52.744304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9079736-4a9b-4f23-ba9a-d30490eaea26 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:52.744604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9079736-4a9b-4f23-ba9a-d30490eaea26 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:52.744888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9079736-4a9b-4f23-ba9a-d30490eaea26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:52.747641 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:52.751776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5a3271e3-f709-4055-8585-f40d797564d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0d3f805c35448a194bba74d75ba2d14'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:52.782903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5a3271e3-f709-4055-8585-f40d797564d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0d3f805c35448a194bba74d75ba2d14'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:52.784402 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5a3271e3-f709-4055-8585-f40d797564d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0d3f805c35448a194bba74d75ba2d14'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:52.793679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-e9079736-4a9b-4f23-ba9a-d30490eaea26 None None] MFA Rules not processed for user `619e3bec841343c5ad144f1d4e4d436d`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:52.797632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e9079736-4a9b-4f23-ba9a-d30490eaea26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:52.813740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5a3271e3-f709-4055-8585-f40d797564d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0d3f805c35448a194bba74d75ba2d14'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:52.842071 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 826/1651] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:51:52 2022] DELETE /identity/v3/users/d0d3f805c35448a194bba74d75ba2d14 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:51:52.846684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6d293-480d-40a5-8dec-644d45c047f5 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:52.847080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6d293-480d-40a5-8dec-644d45c047f5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:52.847238 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6d293-480d-40a5-8dec-644d45c047f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:52.892898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f4f6d293-480d-40a5-8dec-644d45c047f5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:52.899660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f4f6d293-480d-40a5-8dec-644d45c047f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:52.970783 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:53.042565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 826/1652] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Jul 14 17:51:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:51:53.075303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 827/1653] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:53.196403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9079736-4a9b-4f23-ba9a-d30490eaea26 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:53.197826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e9079736-4a9b-4f23-ba9a-d30490eaea26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:53.353086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce06e6-b031-4a93-ad98-b6353fbee844 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:53.353375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce06e6-b031-4a93-ad98-b6353fbee844 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:53.353662 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce06e6-b031-4a93-ad98-b6353fbee844 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:53.355470 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 828/1654] 10.222.0.64 () {54 vars in 960 bytes} [Thu Jul 14 17:51:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:51:53.362337 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab438da-30e8-4e65-9a3b-aaf880f27159 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:53.362337 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab438da-30e8-4e65-9a3b-aaf880f27159 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:53.362337 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab438da-30e8-4e65-9a3b-aaf880f27159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:53.369553 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9079736-4a9b-4f23-ba9a-d30490eaea26 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:53.411431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-7ab438da-30e8-4e65-9a3b-aaf880f27159 None None] MFA Rules not processed for user `619e3bec841343c5ad144f1d4e4d436d`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:53.418808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7ab438da-30e8-4e65-9a3b-aaf880f27159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:53.544770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32d770-5176-4f04-8305-27bc2d249b96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:53.545674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32d770-5176-4f04-8305-27bc2d249b96 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:53.546061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32d770-5176-4f04-8305-27bc2d249b96 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:53.546439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32d770-5176-4f04-8305-27bc2d249b96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:53.584792 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32d770-5176-4f04-8305-27bc2d249b96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:53.588277 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a32d770-5176-4f04-8305-27bc2d249b96 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:53.619623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 827/1655] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:53.621426 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 829/1656] 10.222.0.64 () {58 vars in 1057 bytes} [Thu Jul 14 17:51:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:53.632077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32d770-5176-4f04-8305-27bc2d249b96 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:53.660205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32d770-5176-4f04-8305-27bc2d249b96 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:53.688603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accadb18-0bb7-4458-b5c7-d1f74c5b304e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:53.691966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accadb18-0bb7-4458-b5c7-d1f74c5b304e service placement] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:53.692386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accadb18-0bb7-4458-b5c7-d1f74c5b304e service placement] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:53.693040 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accadb18-0bb7-4458-b5c7-d1f74c5b304e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:53.726763 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accadb18-0bb7-4458-b5c7-d1f74c5b304e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:53.729194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accadb18-0bb7-4458-b5c7-d1f74c5b304e service placement] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:53.776869 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 828/1657] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:53.975194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab438da-30e8-4e65-9a3b-aaf880f27159 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:53.977157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7ab438da-30e8-4e65-9a3b-aaf880f27159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:54.000056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab438da-30e8-4e65-9a3b-aaf880f27159 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:54.019714 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:54.045170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:54.045783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:54.046305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6 admin admin] PATH_INFO: `/v3/projects/1515b50247e84ea9ba5a42cf4cfdf9c2` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:54.052215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1515b50247e84ea9ba5a42cf4cfdf9c2)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:54.053576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: WARNING py.warnings [None req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 17:51:54.053576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: warnings.warn(msg) Jul 14 17:51:54.053576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: Jul 14 17:51:54.059335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:54.195064 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1515b50247e84ea9ba5a42cf4cfdf9c2', 'request_id': 'req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:54.212464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1515b50247e84ea9ba5a42cf4cfdf9c2', 'request_id': 'req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:54.242881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1515b50247e84ea9ba5a42cf4cfdf9c2', 'request_id': 'req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:54.255375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:54.267726 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6 admin admin] The token cache is being invalidate because project 1515b50247e84ea9ba5a42cf4cfdf9c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:51:54.268044 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:54.275585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-88d5a30a-0d2d-4d70-b1fc-048d84be56a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:54.283264 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4177a-c081-4a9e-998f-e392a86f067d None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:54.283612 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4177a-c081-4a9e-998f-e392a86f067d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:54.283911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e4177a-c081-4a9e-998f-e392a86f067d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:54.285276 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 830/1658] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:51:53 2022] DELETE /identity/v3/projects/1515b50247e84ea9ba5a42cf4cfdf9c2 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:54.328213 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-94e4177a-c081-4a9e-998f-e392a86f067d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:54.329728 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-94e4177a-c081-4a9e-998f-e392a86f067d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:54.448815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e9aa9-0caa-4ba9-8ee6-8e7f0cecba4e None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:54.450371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e9aa9-0caa-4ba9-8ee6-8e7f0cecba4e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:54.450691 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e9aa9-0caa-4ba9-8ee6-8e7f0cecba4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:54.455141 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 829/1659] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:54.475280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-e93e9aa9-0caa-4ba9-8ee6-8e7f0cecba4e None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:54.476806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e93e9aa9-0caa-4ba9-8ee6-8e7f0cecba4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:54.491645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4177a-c081-4a9e-998f-e392a86f067d None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:54.502239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 831/1660] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:54.502762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e4177a-c081-4a9e-998f-e392a86f067d None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:54.514928 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710950a4-20fb-421e-8bec-9ad18dd34583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:54.515897 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710950a4-20fb-421e-8bec-9ad18dd34583 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:54.517602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710950a4-20fb-421e-8bec-9ad18dd34583 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:54.518066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710950a4-20fb-421e-8bec-9ad18dd34583 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:54.518984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710950a4-20fb-421e-8bec-9ad18dd34583 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:51:54.519438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710950a4-20fb-421e-8bec-9ad18dd34583 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:54.521296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710950a4-20fb-421e-8bec-9ad18dd34583 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:54.549056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e9aa9-0caa-4ba9-8ee6-8e7f0cecba4e None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:54.560425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e9aa9-0caa-4ba9-8ee6-8e7f0cecba4e None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:54.561444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 830/1661] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:51:54.566463 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec826d-4d44-498e-95b8-2d1ece5ac1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:54.567189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec826d-4d44-498e-95b8-2d1ece5ac1bd admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:54.567460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec826d-4d44-498e-95b8-2d1ece5ac1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:54.567739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec826d-4d44-498e-95b8-2d1ece5ac1bd admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:54.569032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec826d-4d44-498e-95b8-2d1ece5ac1bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:54.571355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec826d-4d44-498e-95b8-2d1ece5ac1bd admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:54.617206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710950a4-20fb-421e-8bec-9ad18dd34583 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:54.624440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 832/1662] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:54 2022] POST /identity/v3/projects => generated 379 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:54.633448 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710950a4-20fb-421e-8bec-9ad18dd34583 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:54.654925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:54.816734 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59659c0-0f5c-4f96-b0d2-9c9ab3c326ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:54.817551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59659c0-0f5c-4f96-b0d2-9c9ab3c326ba admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:54.817945 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59659c0-0f5c-4f96-b0d2-9c9ab3c326ba admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:54.818360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59659c0-0f5c-4f96-b0d2-9c9ab3c326ba admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:54.819455 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59659c0-0f5c-4f96-b0d2-9c9ab3c326ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:54.827030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59659c0-0f5c-4f96-b0d2-9c9ab3c326ba admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:54.909203 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec826d-4d44-498e-95b8-2d1ece5ac1bd admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:54.910233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 831/1663] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:51:54 2022] POST /identity/v3/users => generated 290 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:54.931505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec826d-4d44-498e-95b8-2d1ece5ac1bd admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:54.952752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e6c26-5c14-443e-bb2c-eaaff00a573e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:54.953796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e6c26-5c14-443e-bb2c-eaaff00a573e admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:54.954295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e6c26-5c14-443e-bb2c-eaaff00a573e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:54.954891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e6c26-5c14-443e-bb2c-eaaff00a573e admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:54.962166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e6c26-5c14-443e-bb2c-eaaff00a573e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:54.964184 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e6c26-5c14-443e-bb2c-eaaff00a573e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:54.990996 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 833/1664] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:54.999659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59659c0-0f5c-4f96-b0d2-9c9ab3c326ba admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:55.015212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59659c0-0f5c-4f96-b0d2-9c9ab3c326ba admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:55.033523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68dbf8-e5db-4b07-99a2-23da3e647ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:55.035244 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68dbf8-e5db-4b07-99a2-23da3e647ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:55.036845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68dbf8-e5db-4b07-99a2-23da3e647ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:55.037766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68dbf8-e5db-4b07-99a2-23da3e647ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:55.041017 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68dbf8-e5db-4b07-99a2-23da3e647ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:55.045332 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68dbf8-e5db-4b07-99a2-23da3e647ff2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:55.074284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e6c26-5c14-443e-bb2c-eaaff00a573e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:55.080163 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 832/1665] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:55.086954 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e6c26-5c14-443e-bb2c-eaaff00a573e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:55.093182 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d96768-8918-43ed-88c4-920afcc81c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:55.094400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d96768-8918-43ed-88c4-920afcc81c8a admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:55.094812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d96768-8918-43ed-88c4-920afcc81c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:55.095193 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d96768-8918-43ed-88c4-920afcc81c8a admin admin] PATH_INFO: `/v3/projects/47e3c3d4078b432cb23ba7b22add29eb/users/a4c232b9bc6b4592b2af30b46754f5d4/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:55.140270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d96768-8918-43ed-88c4-920afcc81c8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=47e3c3d4078b432cb23ba7b22add29eb, user_id=a4c232b9bc6b4592b2af30b46754f5d4, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:55.143458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d96768-8918-43ed-88c4-920afcc81c8a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:55.209558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 834/1666] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:55 2022] PUT /identity/v3/projects/47e3c3d4078b432cb23ba7b22add29eb/users/a4c232b9bc6b4592b2af30b46754f5d4/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:51:55.211502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1db2e-beab-4388-9bd8-8f8020836908 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:55.212061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1db2e-beab-4388-9bd8-8f8020836908 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:55.212585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1db2e-beab-4388-9bd8-8f8020836908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:55.272821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-66c1db2e-beab-4388-9bd8-8f8020836908 None None] MFA Rules not processed for user `a4c232b9bc6b4592b2af30b46754f5d4`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:55.274206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-66c1db2e-beab-4388-9bd8-8f8020836908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:55.434244 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 833/1667] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:51:55.438983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d96768-8918-43ed-88c4-920afcc81c8a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:55.605803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d96768-8918-43ed-88c4-920afcc81c8a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:55.611643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e18f2-da42-414d-a405-c3add7089f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:55.612636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e18f2-da42-414d-a405-c3add7089f34 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:55.613112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e18f2-da42-414d-a405-c3add7089f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:55.613559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e18f2-da42-414d-a405-c3add7089f34 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:55.614719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e18f2-da42-414d-a405-c3add7089f34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:55.618116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e18f2-da42-414d-a405-c3add7089f34 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:55.685808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1db2e-beab-4388-9bd8-8f8020836908 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:55.698055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 835/1668] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:55 2022] POST /identity/v3/projects => generated 379 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:55.719719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1db2e-beab-4388-9bd8-8f8020836908 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:55.828953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af3ddf-49b3-4bf9-8445-51baa0fee61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:55.829662 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af3ddf-49b3-4bf9-8445-51baa0fee61a admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:55.829971 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af3ddf-49b3-4bf9-8445-51baa0fee61a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:55.830364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af3ddf-49b3-4bf9-8445-51baa0fee61a admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:55.831192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3af3ddf-49b3-4bf9-8445-51baa0fee61a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:55.838224 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3af3ddf-49b3-4bf9-8445-51baa0fee61a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:55.875045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 834/1669] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:51:55 2022] POST /identity/v3/users => generated 296 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:55.879150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e18f2-da42-414d-a405-c3add7089f34 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:55.895294 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e18f2-da42-414d-a405-c3add7089f34 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:55.901581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499012ed-ec61-4dfd-aaa2-f019998bda9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:55.902571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499012ed-ec61-4dfd-aaa2-f019998bda9d admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:55.903020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499012ed-ec61-4dfd-aaa2-f019998bda9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:55.903410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499012ed-ec61-4dfd-aaa2-f019998bda9d admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:55.913108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499012ed-ec61-4dfd-aaa2-f019998bda9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:55.918052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499012ed-ec61-4dfd-aaa2-f019998bda9d admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:55.947889 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 836/1670] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:55 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:55.952180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af3ddf-49b3-4bf9-8445-51baa0fee61a admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:55.965456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af3ddf-49b3-4bf9-8445-51baa0fee61a admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:55.969234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338c349-1d42-4b20-ad88-4bb84291725c None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:55.974680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338c349-1d42-4b20-ad88-4bb84291725c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:55.975520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3338c349-1d42-4b20-ad88-4bb84291725c None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:55.979731 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 837/1671] 10.222.0.64 () {54 vars in 921 bytes} [Thu Jul 14 17:51:55 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:51:55.985542 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650f843-24ce-4f07-9d91-accc789a09d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:55.986481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650f843-24ce-4f07-9d91-accc789a09d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:55.986767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650f843-24ce-4f07-9d91-accc789a09d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:55.989158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5650f843-24ce-4f07-9d91-accc789a09d2 admin admin] PATH_INFO: `/v3/projects/7c60a51fd76a4b16a9fba425c9daf02c/users/bc43823b53d143fb9a4f7db43b7c1fea/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:55.996911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:56.029151 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650f843-24ce-4f07-9d91-accc789a09d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c60a51fd76a4b16a9fba425c9daf02c, user_id=bc43823b53d143fb9a4f7db43b7c1fea, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:56.032372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5650f843-24ce-4f07-9d91-accc789a09d2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:56.039397 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4dcab-b0c7-4ab0-9bd6-07905f70ef3e None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:56.039689 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4dcab-b0c7-4ab0-9bd6-07905f70ef3e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:56.040629 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd4dcab-b0c7-4ab0-9bd6-07905f70ef3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:56.076180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 835/1672] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:55 2022] PUT /identity/v3/projects/7c60a51fd76a4b16a9fba425c9daf02c/users/bc43823b53d143fb9a4f7db43b7c1fea/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:56.080869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-0cd4dcab-b0c7-4ab0-9bd6-07905f70ef3e None None] MFA Rules not processed for user `619e3bec841343c5ad144f1d4e4d436d`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:56.081282 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650f843-24ce-4f07-9d91-accc789a09d2 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:56.082079 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0cd4dcab-b0c7-4ab0-9bd6-07905f70ef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:56.233961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5650f843-24ce-4f07-9d91-accc789a09d2 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:56.242968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 838/1673] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Jul 14 17:51:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:56.247342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816b9e8-4d54-40e1-82ff-39dfeb2933ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:56.248204 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816b9e8-4d54-40e1-82ff-39dfeb2933ef admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:56.248594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816b9e8-4d54-40e1-82ff-39dfeb2933ef admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:56.249058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816b9e8-4d54-40e1-82ff-39dfeb2933ef admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:56.249911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816b9e8-4d54-40e1-82ff-39dfeb2933ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:56.262263 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816b9e8-4d54-40e1-82ff-39dfeb2933ef admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:56.281669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 836/1674] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:56.281910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4dcab-b0c7-4ab0-9bd6-07905f70ef3e None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:56.293261 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:56.307054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd4dcab-b0c7-4ab0-9bd6-07905f70ef3e None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:56.313586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66198566-37d1-411e-a239-635812962b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:56.314874 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66198566-37d1-411e-a239-635812962b76 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:56.315381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66198566-37d1-411e-a239-635812962b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:56.339645 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66198566-37d1-411e-a239-635812962b76 admin admin] PATH_INFO: `/v3/projects/7c60a51fd76a4b16a9fba425c9daf02c/users/bc43823b53d143fb9a4f7db43b7c1fea/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:56.344944 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66198566-37d1-411e-a239-635812962b76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c60a51fd76a4b16a9fba425c9daf02c, user_id=bc43823b53d143fb9a4f7db43b7c1fea, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:56.356357 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66198566-37d1-411e-a239-635812962b76 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:56.373932 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 839/1675] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:56 2022] PUT /identity/v3/projects/7c60a51fd76a4b16a9fba425c9daf02c/users/bc43823b53d143fb9a4f7db43b7c1fea/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:56.384769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993ae8e-e534-480c-94a3-db5e664ad352 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:56.385071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993ae8e-e534-480c-94a3-db5e664ad352 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:56.385689 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7993ae8e-e534-480c-94a3-db5e664ad352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:56.453964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-7993ae8e-e534-480c-94a3-db5e664ad352 None None] MFA Rules not processed for user `bc43823b53d143fb9a4f7db43b7c1fea`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:56.457542 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7993ae8e-e534-480c-94a3-db5e664ad352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:56.473554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66198566-37d1-411e-a239-635812962b76 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:56.476668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-66198566-37d1-411e-a239-635812962b76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:56.667145 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66198566-37d1-411e-a239-635812962b76 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:56.677095 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f59254-3485-42da-b1b2-410c53f56373 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:56.678441 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f59254-3485-42da-b1b2-410c53f56373 service placement] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:56.678972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f59254-3485-42da-b1b2-410c53f56373 service placement] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:56.679455 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f59254-3485-42da-b1b2-410c53f56373 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:56.696186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 837/1676] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:56.697197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993ae8e-e534-480c-94a3-db5e664ad352 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:56.698752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f59254-3485-42da-b1b2-410c53f56373 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:56.701217 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f59254-3485-42da-b1b2-410c53f56373 service placement] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:56.706560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7993ae8e-e534-480c-94a3-db5e664ad352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:56.719002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 840/1677] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:51:56.719892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f59254-3485-42da-b1b2-410c53f56373 service placement] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:56.721280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d9f59254-3485-42da-b1b2-410c53f56373 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:56.885653 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7993ae8e-e534-480c-94a3-db5e664ad352 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:56.897592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:56.903599 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f59254-3485-42da-b1b2-410c53f56373 service placement] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:56.904354 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:56.904858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:56.907425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] PATH_INFO: `/v3/projects/ceae870c225d45cca6d22ed0f5b072fe` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:56.909717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceae870c225d45cca6d22ed0f5b072fe)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:56.912031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:56.924860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:56.931590 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd94689d-0622-48ff-af55-09e595e5cc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:56.933664 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94689d-0622-48ff-af55-09e595e5cc62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:56.934522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94689d-0622-48ff-af55-09e595e5cc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:56.935911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd94689d-0622-48ff-af55-09e595e5cc62 admin admin] PATH_INFO: `/v3/users/2fff9718251f49fb8b532b312fb17074` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:56.938680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94689d-0622-48ff-af55-09e595e5cc62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fff9718251f49fb8b532b312fb17074)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:56.946253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd94689d-0622-48ff-af55-09e595e5cc62 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:57.056959 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceae870c225d45cca6d22ed0f5b072fe', 'request_id': 'req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:57.086056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceae870c225d45cca6d22ed0f5b072fe', 'request_id': 'req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:57.093265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceae870c225d45cca6d22ed0f5b072fe', 'request_id': 'req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:57.113403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-dd94689d-0622-48ff-af55-09e595e5cc62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fff9718251f49fb8b532b312fb17074'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:57.121186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] The token cache is being invalidate because project ceae870c225d45cca6d22ed0f5b072fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:51:57.121794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:57.122751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:57.125627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 838/1678] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:51:56 2022] DELETE /identity/v3/projects/ceae870c225d45cca6d22ed0f5b072fe => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:57.126795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:57.129751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:57.163581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-dd94689d-0622-48ff-af55-09e595e5cc62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fff9718251f49fb8b532b312fb17074'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:57.197675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-dd94689d-0622-48ff-af55-09e595e5cc62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fff9718251f49fb8b532b312fb17074'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:57.199484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-dd94689d-0622-48ff-af55-09e595e5cc62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fff9718251f49fb8b532b312fb17074'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:57.215469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeeeaaa-62bd-41d2-9f3c-047824dc1167 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:57.224359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05bef5a-ce1a-4444-b136-9ada3dc6c7ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:57.225487 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bef5a-ce1a-4444-b136-9ada3dc6c7ac service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:57.225884 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bef5a-ce1a-4444-b136-9ada3dc6c7ac service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:57.226274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05bef5a-ce1a-4444-b136-9ada3dc6c7ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:57.227745 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-dd94689d-0622-48ff-af55-09e595e5cc62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fff9718251f49fb8b532b312fb17074'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:57.229605 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c05bef5a-ce1a-4444-b136-9ada3dc6c7ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:57.242203 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:57.268728 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 841/1679] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:51:56 2022] DELETE /identity/v3/users/2fff9718251f49fb8b532b312fb17074 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:57.271974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c1871-56b5-4347-a032-b93909a4c3c4 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:57.272301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c1871-56b5-4347-a032-b93909a4c3c4 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:57.272770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c1871-56b5-4347-a032-b93909a4c3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:57.371917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-3d0c1871-56b5-4347-a032-b93909a4c3c4 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:57.374567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3d0c1871-56b5-4347-a032-b93909a4c3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:57.406729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05bef5a-ce1a-4444-b136-9ada3dc6c7ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:57.412074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05bef5a-ce1a-4444-b136-9ada3dc6c7ac service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:57.461192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 839/1680] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:57 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:57.578195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 842/1681] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:57.586179 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf669f2-2ef2-4bc6-af25-a5f02079c1b2 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:57.587646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf669f2-2ef2-4bc6-af25-a5f02079c1b2 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:57.588666 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf669f2-2ef2-4bc6-af25-a5f02079c1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:57.632752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-acf669f2-2ef2-4bc6-af25-a5f02079c1b2 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:57.635142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-acf669f2-2ef2-4bc6-af25-a5f02079c1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:57.663719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 840/1682] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:57.666765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c1871-56b5-4347-a032-b93909a4c3c4 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:57.682469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c1871-56b5-4347-a032-b93909a4c3c4 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:57.698828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137e595-96d3-4095-927c-f04232dffa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:57.699692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137e595-96d3-4095-927c-f04232dffa8c admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:57.700005 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137e595-96d3-4095-927c-f04232dffa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:57.700295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9137e595-96d3-4095-927c-f04232dffa8c admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:57.701047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137e595-96d3-4095-927c-f04232dffa8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:51:57.701362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137e595-96d3-4095-927c-f04232dffa8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:57.702979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9137e595-96d3-4095-927c-f04232dffa8c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:57.768959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 843/1683] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:51:57.769446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf669f2-2ef2-4bc6-af25-a5f02079c1b2 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:57.784685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf669f2-2ef2-4bc6-af25-a5f02079c1b2 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:57.802519 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde72702-4608-47b1-8dc3-6b877d2b61d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:57.804193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde72702-4608-47b1-8dc3-6b877d2b61d9 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:57.804643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde72702-4608-47b1-8dc3-6b877d2b61d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:57.805075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde72702-4608-47b1-8dc3-6b877d2b61d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:57.805924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde72702-4608-47b1-8dc3-6b877d2b61d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:57.809217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde72702-4608-47b1-8dc3-6b877d2b61d9 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:57.877755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 841/1684] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:57 2022] POST /identity/v3/projects => generated 403 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:57.877984 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137e595-96d3-4095-927c-f04232dffa8c admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:57.898284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9137e595-96d3-4095-927c-f04232dffa8c admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:57.921161 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90978764-64f2-44aa-98ad-5974de314c55 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:57.922322 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90978764-64f2-44aa-98ad-5974de314c55 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:57.922765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90978764-64f2-44aa-98ad-5974de314c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:58.022316 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-90978764-64f2-44aa-98ad-5974de314c55 None None] MFA Rules not processed for user `be1201ac0aa7484f96e271410342a6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:58.023535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-90978764-64f2-44aa-98ad-5974de314c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:58.037887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545e8e1-18fe-4213-aff2-ca6e5f2f30f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:58.038869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545e8e1-18fe-4213-aff2-ca6e5f2f30f2 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:58.039576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545e8e1-18fe-4213-aff2-ca6e5f2f30f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:58.040121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545e8e1-18fe-4213-aff2-ca6e5f2f30f2 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:58.040940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545e8e1-18fe-4213-aff2-ca6e5f2f30f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:58.043801 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545e8e1-18fe-4213-aff2-ca6e5f2f30f2 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:58.047025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:58.108655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 844/1685] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:51:57 2022] POST /identity/v3/users => generated 302 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:58.109271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545e8e1-18fe-4213-aff2-ca6e5f2f30f2 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:58.111518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5545e8e1-18fe-4213-aff2-ca6e5f2f30f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:58.133596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545e8e1-18fe-4213-aff2-ca6e5f2f30f2 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:58.144232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:58.147561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:58.148109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:58.148929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] PATH_INFO: `/v3/projects/b7ee3a419a5c4b8d81d103e92a77c4bf` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:58.152660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7ee3a419a5c4b8d81d103e92a77c4bf)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:58.155183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:58.248906 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 842/1686] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:58.249601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ee3a419a5c4b8d81d103e92a77c4bf', 'request_id': 'req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:58.249708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90978764-64f2-44aa-98ad-5974de314c55 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:58.250139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ee3a419a5c4b8d81d103e92a77c4bf', 'request_id': 'req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:58.259387 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ee3a419a5c4b8d81d103e92a77c4bf', 'request_id': 'req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:58.269481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90978764-64f2-44aa-98ad-5974de314c55 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:58.276517 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] The token cache is being invalidate because project b7ee3a419a5c4b8d81d103e92a77c4bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:51:58.277020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:58.278460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:58.280534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:58.300032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 845/1687] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:51:58 2022] DELETE /identity/v3/projects/b7ee3a419a5c4b8d81d103e92a77c4bf => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:58.301646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:58.303579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:58.390637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d09b410-63ab-4c4e-a8ed-ec624f9a39c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:58.391326 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09b410-63ab-4c4e-a8ed-ec624f9a39c6 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:58.391613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09b410-63ab-4c4e-a8ed-ec624f9a39c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:58.391891 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d09b410-63ab-4c4e-a8ed-ec624f9a39c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:58.392693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09b410-63ab-4c4e-a8ed-ec624f9a39c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:58.394320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d09b410-63ab-4c4e-a8ed-ec624f9a39c6 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:58.427313 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 843/1688] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:58.433440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b501ac9-778f-4873-b945-9a7c56a7766f None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:58.433440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b501ac9-778f-4873-b945-9a7c56a7766f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:58.433728 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b501ac9-778f-4873-b945-9a7c56a7766f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:58.445241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbeab9e-e963-45ae-84cd-5990d51fa6fb admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:58.461413 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e63394-e9dd-44c6-b243-2e1b0ad63f6a tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc43823b53d143fb9a4f7db43b7c1fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c60a51fd76a4b16a9fba425c9daf02c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:58.462058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e63394-e9dd-44c6-b243-2e1b0ad63f6a tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:58.462351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e63394-e9dd-44c6-b243-2e1b0ad63f6a tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:58.462632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e63394-e9dd-44c6-b243-2e1b0ad63f6a tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:58.463383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e63394-e9dd-44c6-b243-2e1b0ad63f6a tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:51:58.463698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e63394-e9dd-44c6-b243-2e1b0ad63f6a tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:58.465356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e63394-e9dd-44c6-b243-2e1b0ad63f6a tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:58.471565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-6b501ac9-778f-4873-b945-9a7c56a7766f None None] MFA Rules not processed for user `be1201ac0aa7484f96e271410342a6b9`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:58.474231 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6b501ac9-778f-4873-b945-9a7c56a7766f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:58.492782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 846/1689] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:51:58.493792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e63394-e9dd-44c6-b243-2e1b0ad63f6a tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:58.499224 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c6e63394-e9dd-44c6-b243-2e1b0ad63f6a tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:58.526085 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e63394-e9dd-44c6-b243-2e1b0ad63f6a tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:58.537435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc50159-64da-41f4-b1f7-3316b9ddc299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:58.538346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc50159-64da-41f4-b1f7-3316b9ddc299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:58.538636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc50159-64da-41f4-b1f7-3316b9ddc299 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:58.539186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc50159-64da-41f4-b1f7-3316b9ddc299 admin admin] PATH_INFO: `/v3/users/05fe506a87a443dab657b8a0a48f9f6a` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:58.540947 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc50159-64da-41f4-b1f7-3316b9ddc299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05fe506a87a443dab657b8a0a48f9f6a)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:58.546174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc50159-64da-41f4-b1f7-3316b9ddc299 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:58.614253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 844/1690] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:58.614253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b501ac9-778f-4873-b945-9a7c56a7766f None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:58.614253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6b501ac9-778f-4873-b945-9a7c56a7766f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:58.640625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b501ac9-778f-4873-b945-9a7c56a7766f None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:58.649000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42e3e-a459-4d43-a2cf-daa40159d88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:58.650452 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42e3e-a459-4d43-a2cf-daa40159d88a admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:58.651004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42e3e-a459-4d43-a2cf-daa40159d88a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:58.651416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea42e3e-a459-4d43-a2cf-daa40159d88a admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:58.653441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea42e3e-a459-4d43-a2cf-daa40159d88a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:58.654042 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ebc50159-64da-41f4-b1f7-3316b9ddc299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05fe506a87a443dab657b8a0a48f9f6a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:58.655880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea42e3e-a459-4d43-a2cf-daa40159d88a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:58.675028 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 845/1691] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:51:58 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:51:58.680188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42e3e-a459-4d43-a2cf-daa40159d88a admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:58.688994 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ebc50159-64da-41f4-b1f7-3316b9ddc299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05fe506a87a443dab657b8a0a48f9f6a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:58.697032 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea42e3e-a459-4d43-a2cf-daa40159d88a admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:58.705629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92672d-c56c-4744-9cd8-7d55839c4249 tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc43823b53d143fb9a4f7db43b7c1fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c60a51fd76a4b16a9fba425c9daf02c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:58.706461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92672d-c56c-4744-9cd8-7d55839c4249 tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:58.707763 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92672d-c56c-4744-9cd8-7d55839c4249 tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:58.708297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92672d-c56c-4744-9cd8-7d55839c4249 tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:58.710540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ebc50159-64da-41f4-b1f7-3316b9ddc299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05fe506a87a443dab657b8a0a48f9f6a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:58.711545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ebc50159-64da-41f4-b1f7-3316b9ddc299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05fe506a87a443dab657b8a0a48f9f6a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:58.717708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92672d-c56c-4744-9cd8-7d55839c4249 tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:58.721354 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92672d-c56c-4744-9cd8-7d55839c4249 tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:58.724479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ebc50159-64da-41f4-b1f7-3316b9ddc299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05fe506a87a443dab657b8a0a48f9f6a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:58.746367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 847/1692] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:51:58 2022] DELETE /identity/v3/users/05fe506a87a443dab657b8a0a48f9f6a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:58.747936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc50159-64da-41f4-b1f7-3316b9ddc299 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:58.750976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ebc50159-64da-41f4-b1f7-3316b9ddc299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:58.781358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 846/1693] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:58 2022] POST /identity/v3/projects => generated 353 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:51:58.782359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92672d-c56c-4744-9cd8-7d55839c4249 tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:58.785484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5a92672d-c56c-4744-9cd8-7d55839c4249 tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:58.827728 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:58.915531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92672d-c56c-4744-9cd8-7d55839c4249 tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:58.922359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65779d35-f4e8-4081-9d60-f647fe4407cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:58.924549 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65779d35-f4e8-4081-9d60-f647fe4407cf admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:58.925454 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65779d35-f4e8-4081-9d60-f647fe4407cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:58.925751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65779d35-f4e8-4081-9d60-f647fe4407cf admin admin] PATH_INFO: `/v3/projects/27955b7afd6f493382198cdfc7bcb681/users/de4e509aa41941248b8273ed06f7e23e/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:58.953573 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:58.956772 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc50159-64da-41f4-b1f7-3316b9ddc299 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:58.961132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65779d35-f4e8-4081-9d60-f647fe4407cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=27955b7afd6f493382198cdfc7bcb681, user_id=de4e509aa41941248b8273ed06f7e23e, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:58.966962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65779d35-f4e8-4081-9d60-f647fe4407cf admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:58.999275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 847/1694] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:51:58 2022] PUT /identity/v3/projects/27955b7afd6f493382198cdfc7bcb681/users/de4e509aa41941248b8273ed06f7e23e/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:59.000589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65779d35-f4e8-4081-9d60-f647fe4407cf admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:59.008868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-65779d35-f4e8-4081-9d60-f647fe4407cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:59.133687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee9a8f-31e9-4e01-b796-de3c1e531b03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:59.134869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee9a8f-31e9-4e01-b796-de3c1e531b03 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:59.135334 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee9a8f-31e9-4e01-b796-de3c1e531b03 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:59.135790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee9a8f-31e9-4e01-b796-de3c1e531b03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:59.138302 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-81ee9a8f-31e9-4e01-b796-de3c1e531b03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:59.202462 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65779d35-f4e8-4081-9d60-f647fe4407cf admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:59.212373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc286b-2ce7-4adc-b423-5590927b40bb tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc43823b53d143fb9a4f7db43b7c1fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c60a51fd76a4b16a9fba425c9daf02c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:59.213102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc286b-2ce7-4adc-b423-5590927b40bb tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:59.213462 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc286b-2ce7-4adc-b423-5590927b40bb tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:59.214162 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc286b-2ce7-4adc-b423-5590927b40bb tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:59.214633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc286b-2ce7-4adc-b423-5590927b40bb tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:51:59.215107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc286b-2ce7-4adc-b423-5590927b40bb tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:59.216759 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc286b-2ce7-4adc-b423-5590927b40bb tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:59.247981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 848/1695] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:51:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:51:59.249980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0b86f-6301-4d0a-b800-fec4b61d254c None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:59.250276 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0b86f-6301-4d0a-b800-fec4b61d254c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:59.250624 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef0b86f-6301-4d0a-b800-fec4b61d254c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:59.279275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee9a8f-31e9-4e01-b796-de3c1e531b03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:59.280745 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ee9a8f-31e9-4e01-b796-de3c1e531b03 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:59.309068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4ef0b86f-6301-4d0a-b800-fec4b61d254c None None] MFA Rules not processed for user `de4e509aa41941248b8273ed06f7e23e`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:59.309855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 848/1696] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:51:58 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:51:59.310730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4ef0b86f-6301-4d0a-b800-fec4b61d254c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:59.319893 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee9a8f-31e9-4e01-b796-de3c1e531b03 service nova] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:59.459833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 849/1697] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:51:59.463585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0b86f-6301-4d0a-b800-fec4b61d254c None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:59.464724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4ef0b86f-6301-4d0a-b800-fec4b61d254c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:59.512834 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee9a8f-31e9-4e01-b796-de3c1e531b03 service nova] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:59.528757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc43823b53d143fb9a4f7db43b7c1fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7c60a51fd76a4b16a9fba425c9daf02c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:59.529205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:59.529913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:59.530222 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:59.530604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] PATH_INFO: `/v3/projects/b7ffad80f1204c3c93c41b2059561c97` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:59.536834 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b7ffad80f1204c3c93c41b2059561c97)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:59.539524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:59.625448 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef0b86f-6301-4d0a-b800-fec4b61d254c None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:51:59.639284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f36cc5-0201-49ea-b40a-5e52ded8e72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:51:59.640222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f36cc5-0201-49ea-b40a-5e52ded8e72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:59.640508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f36cc5-0201-49ea-b40a-5e52ded8e72a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:59.640791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f36cc5-0201-49ea-b40a-5e52ded8e72a admin admin] PATH_INFO: `/v3/projects/b5852053939d420c8ee0d372826597b7` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:59.646160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f36cc5-0201-49ea-b40a-5e52ded8e72a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5852053939d420c8ee0d372826597b7)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:51:59.652880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f36cc5-0201-49ea-b40a-5e52ded8e72a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:51:59.692183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:51:59.704361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7ffad80f1204c3c93c41b2059561c97', 'request_id': 'req-e3cd1073-3e00-464a-817a-88e057e7bf9c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:59.724480 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7ffad80f1204c3c93c41b2059561c97', 'request_id': 'req-e3cd1073-3e00-464a-817a-88e057e7bf9c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:59.737185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7ffad80f1204c3c93c41b2059561c97', 'request_id': 'req-e3cd1073-3e00-464a-817a-88e057e7bf9c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:59.760147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] The token cache is being invalidate because project b7ffad80f1204c3c93c41b2059561c97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:51:59.760616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:59.761388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:59.763081 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 849/1698] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:51:59 2022] DELETE /identity/v3/projects/b7ffad80f1204c3c93c41b2059561c97 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:59.767183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:51:59.768169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:51:59.797015 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-33f36cc5-0201-49ea-b40a-5e52ded8e72a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5852053939d420c8ee0d372826597b7', 'request_id': 'req-33f36cc5-0201-49ea-b40a-5e52ded8e72a'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:59.812116 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-33f36cc5-0201-49ea-b40a-5e52ded8e72a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5852053939d420c8ee0d372826597b7', 'request_id': 'req-33f36cc5-0201-49ea-b40a-5e52ded8e72a'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:59.828272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-33f36cc5-0201-49ea-b40a-5e52ded8e72a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5852053939d420c8ee0d372826597b7', 'request_id': 'req-33f36cc5-0201-49ea-b40a-5e52ded8e72a'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:59.875960 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-33f36cc5-0201-49ea-b40a-5e52ded8e72a admin admin] The token cache is being invalidate because project b5852053939d420c8ee0d372826597b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:51:59.876267 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-33f36cc5-0201-49ea-b40a-5e52ded8e72a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:59.883594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-33f36cc5-0201-49ea-b40a-5e52ded8e72a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:51:59.892363 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 850/1699] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:51:59 2022] DELETE /identity/v3/projects/b5852053939d420c8ee0d372826597b7 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:51:59.895757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815f06c-7c2c-4277-bf2c-648e7e8dffea None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:51:59.896075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815f06c-7c2c-4277-bf2c-648e7e8dffea None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:51:59.898265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2815f06c-7c2c-4277-bf2c-648e7e8dffea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:51:59.957938 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-2815f06c-7c2c-4277-bf2c-648e7e8dffea None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:51:59.971536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-2815f06c-7c2c-4277-bf2c-648e7e8dffea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:00.014559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd1073-3e00-464a-817a-88e057e7bf9c tempest-VolumeQuotaClassesTest-363318302 tempest-VolumeQuotaClassesTest-363318302-project-admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:00.136257 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 851/1700] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:00.141856 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83bda4-e416-4a8c-964f-bf68bb82b494 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:00.142162 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83bda4-e416-4a8c-964f-bf68bb82b494 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:00.142451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83bda4-e416-4a8c-964f-bf68bb82b494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:00.196505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-eb83bda4-e416-4a8c-964f-bf68bb82b494 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:00.199162 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-eb83bda4-e416-4a8c-964f-bf68bb82b494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:00.218689 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 852/1701] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:00.221648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf9ce5-010e-4652-8004-087f3cdf187b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:00.222786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83bda4-e416-4a8c-964f-bf68bb82b494 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:00.223666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf9ce5-010e-4652-8004-087f3cdf187b admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:00.224121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf9ce5-010e-4652-8004-087f3cdf187b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:00.226280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbf9ce5-010e-4652-8004-087f3cdf187b admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:00.227169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf9ce5-010e-4652-8004-087f3cdf187b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:00.230911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbf9ce5-010e-4652-8004-087f3cdf187b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:00.252856 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83bda4-e416-4a8c-964f-bf68bb82b494 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:00.264279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60302d36-efeb-4bfd-a6fd-51c2114a5053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:00.267430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60302d36-efeb-4bfd-a6fd-51c2114a5053 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:00.268104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60302d36-efeb-4bfd-a6fd-51c2114a5053 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:00.268553 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60302d36-efeb-4bfd-a6fd-51c2114a5053 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:00.270973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60302d36-efeb-4bfd-a6fd-51c2114a5053 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:52:00.272687 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60302d36-efeb-4bfd-a6fd-51c2114a5053 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:00.274864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60302d36-efeb-4bfd-a6fd-51c2114a5053 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:00.312345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 850/1702] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:51:59 2022] POST /identity/v3/projects => generated 403 bytes in 546 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:00.315674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf9ce5-010e-4652-8004-087f3cdf187b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:00.317355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 853/1703] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:52:00.318257 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60302d36-efeb-4bfd-a6fd-51c2114a5053 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:00.319200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4bbf9ce5-010e-4652-8004-087f3cdf187b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:00.320655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-60302d36-efeb-4bfd-a6fd-51c2114a5053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:00.341807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60302d36-efeb-4bfd-a6fd-51c2114a5053 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:00.416651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:00.493330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbf9ce5-010e-4652-8004-087f3cdf187b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:00.499037 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea2e5e-052c-4f16-8c3f-b9b32ba8fe9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:00.505491 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea2e5e-052c-4f16-8c3f-b9b32ba8fe9b admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:00.506431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e642a2-3600-415d-bf22-e426d7bb0015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:00.507074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea2e5e-052c-4f16-8c3f-b9b32ba8fe9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:00.507511 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ea2e5e-052c-4f16-8c3f-b9b32ba8fe9b admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:00.507620 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e642a2-3600-415d-bf22-e426d7bb0015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:00.508025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e642a2-3600-415d-bf22-e426d7bb0015 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:00.508404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e642a2-3600-415d-bf22-e426d7bb0015 admin admin] PATH_INFO: `/v3/users/a4c232b9bc6b4592b2af30b46754f5d4` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:00.509226 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea2e5e-052c-4f16-8c3f-b9b32ba8fe9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:00.510671 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e642a2-3600-415d-bf22-e426d7bb0015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4c232b9bc6b4592b2af30b46754f5d4)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:00.516468 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e642a2-3600-415d-bf22-e426d7bb0015 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:00.517403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ea2e5e-052c-4f16-8c3f-b9b32ba8fe9b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:00.532164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:00.600439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 854/1704] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:00 2022] POST /identity/v3/users => generated 308 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:00.602063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea2e5e-052c-4f16-8c3f-b9b32ba8fe9b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:00.635126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ea2e5e-052c-4f16-8c3f-b9b32ba8fe9b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:00.649749 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c21b6a-348e-471b-b228-0ae720791910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:00.650733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c21b6a-348e-471b-b228-0ae720791910 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:00.651147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c21b6a-348e-471b-b228-0ae720791910 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:00.651530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c21b6a-348e-471b-b228-0ae720791910 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:00.652373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c21b6a-348e-471b-b228-0ae720791910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:00.658930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c21b6a-348e-471b-b228-0ae720791910 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:00.664545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-45e642a2-3600-415d-bf22-e426d7bb0015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4c232b9bc6b4592b2af30b46754f5d4'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:00.702027 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-45e642a2-3600-415d-bf22-e426d7bb0015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4c232b9bc6b4592b2af30b46754f5d4'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:00.721460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-45e642a2-3600-415d-bf22-e426d7bb0015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4c232b9bc6b4592b2af30b46754f5d4'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:00.722529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-45e642a2-3600-415d-bf22-e426d7bb0015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4c232b9bc6b4592b2af30b46754f5d4'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:00.723489 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 855/1705] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:00 2022] POST /identity/v3/projects => generated 371 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:00.724582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c21b6a-348e-471b-b228-0ae720791910 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:00.738230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-45e642a2-3600-415d-bf22-e426d7bb0015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4c232b9bc6b4592b2af30b46754f5d4'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:00.764561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 851/1706] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:00 2022] DELETE /identity/v3/users/a4c232b9bc6b4592b2af30b46754f5d4 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:00.764561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422bf7b-d498-4c1b-bef1-85a2ce56bc4e None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:00.764561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422bf7b-d498-4c1b-bef1-85a2ce56bc4e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:00.764561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422bf7b-d498-4c1b-bef1-85a2ce56bc4e None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:00.765683 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 852/1707] 10.222.0.64 () {54 vars in 912 bytes} [Thu Jul 14 17:52:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:52:00.766400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422bf7b-d498-4c1b-bef1-85a2ce56bc4e None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:00.769275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d422bf7b-d498-4c1b-bef1-85a2ce56bc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:00.849026 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c21b6a-348e-471b-b228-0ae720791910 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:00.967616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422bf7b-d498-4c1b-bef1-85a2ce56bc4e None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:00.980720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3060f8-740a-4fc2-8881-211c9fc2d7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:00.981429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3060f8-740a-4fc2-8881-211c9fc2d7af admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:00.981723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3060f8-740a-4fc2-8881-211c9fc2d7af admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:00.982003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3060f8-740a-4fc2-8881-211c9fc2d7af admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:00.982712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3060f8-740a-4fc2-8881-211c9fc2d7af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:00.985200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3060f8-740a-4fc2-8881-211c9fc2d7af admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:01.011482 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1164d0e7-0d81-405d-8fd4-898b4b504129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:01.013399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164d0e7-0d81-405d-8fd4-898b4b504129 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:01.014845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164d0e7-0d81-405d-8fd4-898b4b504129 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:01.017144 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1164d0e7-0d81-405d-8fd4-898b4b504129 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:01.021254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164d0e7-0d81-405d-8fd4-898b4b504129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:01.026075 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 853/1708] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:00 2022] POST /identity/v3/users => generated 286 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:01.027422 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff316e-92b6-43b8-b831-0510fd3f4d4c None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:01.027719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff316e-92b6-43b8-b831-0510fd3f4d4c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:01.028022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bff316e-92b6-43b8-b831-0510fd3f4d4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:01.029500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1164d0e7-0d81-405d-8fd4-898b4b504129 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:01.059698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 856/1709] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:01.063764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303d406-ff6c-4abb-aa46-633f45b201fd None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:01.064108 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303d406-ff6c-4abb-aa46-633f45b201fd None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:01.064401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303d406-ff6c-4abb-aa46-633f45b201fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:01.080835 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-4bff316e-92b6-43b8-b831-0510fd3f4d4c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:01.082149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4bff316e-92b6-43b8-b831-0510fd3f4d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:01.124938 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 854/1710] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:01.128347 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff316e-92b6-43b8-b831-0510fd3f4d4c None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:01.141225 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4bff316e-92b6-43b8-b831-0510fd3f4d4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:01.163837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4303d406-ff6c-4abb-aa46-633f45b201fd None None] MFA Rules not processed for user `a5adec9924444e29b8964f7969452592`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:01.166611 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bff316e-92b6-43b8-b831-0510fd3f4d4c None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:01.166753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4303d406-ff6c-4abb-aa46-633f45b201fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:01.190715 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96baf7-dc4d-4cd4-8d10-7ee626113486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:01.191775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96baf7-dc4d-4cd4-8d10-7ee626113486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:01.192181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96baf7-dc4d-4cd4-8d10-7ee626113486 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:01.192557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96baf7-dc4d-4cd4-8d10-7ee626113486 admin admin] PATH_INFO: `/v3/users/197bd8804cd44905b65bbbc5cb191bfa` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:01.197204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96baf7-dc4d-4cd4-8d10-7ee626113486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=197bd8804cd44905b65bbbc5cb191bfa)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:01.201862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96baf7-dc4d-4cd4-8d10-7ee626113486 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:01.244157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:01.316909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 857/1711] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 17:52:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:01.323767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303d406-ff6c-4abb-aa46-633f45b201fd None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:01.325400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7d96baf7-dc4d-4cd4-8d10-7ee626113486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '197bd8804cd44905b65bbbc5cb191bfa'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:01.347089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303d406-ff6c-4abb-aa46-633f45b201fd None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:01.353102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=78,keystone:DELETE=18,keystone:INSERT=4 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:01.357456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7d96baf7-dc4d-4cd4-8d10-7ee626113486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '197bd8804cd44905b65bbbc5cb191bfa'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:01.357887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7f483-0a99-4700-940e-65ef4ddab426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:01.358579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7f483-0a99-4700-940e-65ef4ddab426 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:01.358994 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7f483-0a99-4700-940e-65ef4ddab426 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:01.359289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe7f483-0a99-4700-940e-65ef4ddab426 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:01.360175 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7f483-0a99-4700-940e-65ef4ddab426 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:01.361889 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe7f483-0a99-4700-940e-65ef4ddab426 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:01.371206 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7d96baf7-dc4d-4cd4-8d10-7ee626113486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '197bd8804cd44905b65bbbc5cb191bfa'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:01.372327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7d96baf7-dc4d-4cd4-8d10-7ee626113486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '197bd8804cd44905b65bbbc5cb191bfa'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:01.383590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 858/1712] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:01.383590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7f483-0a99-4700-940e-65ef4ddab426 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:01.390266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7d96baf7-dc4d-4cd4-8d10-7ee626113486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '197bd8804cd44905b65bbbc5cb191bfa'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:01.398913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 855/1713] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:01 2022] DELETE /identity/v3/users/197bd8804cd44905b65bbbc5cb191bfa => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:01.400073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96baf7-dc4d-4cd4-8d10-7ee626113486 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:01.400281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5fe7f483-0a99-4700-940e-65ef4ddab426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:01.401801 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7d96baf7-dc4d-4cd4-8d10-7ee626113486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:01.434743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe7f483-0a99-4700-940e-65ef4ddab426 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:01.494308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96baf7-dc4d-4cd4-8d10-7ee626113486 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:01.502492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8d11b-bb4a-45c8-8328-db0f45c2c519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:01.503176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8d11b-bb4a-45c8-8328-db0f45c2c519 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:01.503589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8d11b-bb4a-45c8-8328-db0f45c2c519 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:01.503795 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8d11b-bb4a-45c8-8328-db0f45c2c519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:01.508923 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-40c8d11b-bb4a-45c8-8328-db0f45c2c519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:01.555297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09763798-6df3-4981-a72e-535f8beb6523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:01.556691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09763798-6df3-4981-a72e-535f8beb6523 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:01.557251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09763798-6df3-4981-a72e-535f8beb6523 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:01.557729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09763798-6df3-4981-a72e-535f8beb6523 admin admin] PATH_INFO: `/v3/projects/0f8aa7d4bbd64c838eaad1a303e7a573/users/2cab6a8fe9c5400ba13bf02f723d77f2/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:01.594109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09763798-6df3-4981-a72e-535f8beb6523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8aa7d4bbd64c838eaad1a303e7a573, user_id=2cab6a8fe9c5400ba13bf02f723d77f2, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:01.598467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09763798-6df3-4981-a72e-535f8beb6523 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:01.600662 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8d11b-bb4a-45c8-8328-db0f45c2c519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:01.602650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8d11b-bb4a-45c8-8328-db0f45c2c519 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:01.616694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 856/1714] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:01.617878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8d11b-bb4a-45c8-8328-db0f45c2c519 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:01.626238 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 859/1715] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:01 2022] PUT /identity/v3/projects/0f8aa7d4bbd64c838eaad1a303e7a573/users/2cab6a8fe9c5400ba13bf02f723d77f2/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:01.627603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8d11b-bb4a-45c8-8328-db0f45c2c519 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:01.628484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09763798-6df3-4981-a72e-535f8beb6523 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:01.633488 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-09763798-6df3-4981-a72e-535f8beb6523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:01.698562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54baaf48-276f-4fdd-8e41-f91122c43ab2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:01.699246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baaf48-276f-4fdd-8e41-f91122c43ab2 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:01.699531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baaf48-276f-4fdd-8e41-f91122c43ab2 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:01.699809 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baaf48-276f-4fdd-8e41-f91122c43ab2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:01.702229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-54baaf48-276f-4fdd-8e41-f91122c43ab2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:01.716865 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54baaf48-276f-4fdd-8e41-f91122c43ab2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:01.718699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54baaf48-276f-4fdd-8e41-f91122c43ab2 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:01.730429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 857/1716] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:01.731849 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad6f3e-4ac1-446b-b9d2-a8ecb853c4d5 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:01.732172 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad6f3e-4ac1-446b-b9d2-a8ecb853c4d5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:01.732457 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad6f3e-4ac1-446b-b9d2-a8ecb853c4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:01.753460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-64ad6f3e-4ac1-446b-b9d2-a8ecb853c4d5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:01.754695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-64ad6f3e-4ac1-446b-b9d2-a8ecb853c4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:01.768387 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 858/1717] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:01.769627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09763798-6df3-4981-a72e-535f8beb6523 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:01.770424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad6f3e-4ac1-446b-b9d2-a8ecb853c4d5 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:01.772887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-64ad6f3e-4ac1-446b-b9d2-a8ecb853c4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:01.788411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeb4aa-2343-49c7-9e0a-f298aa205280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:01.789116 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeb4aa-2343-49c7-9e0a-f298aa205280 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:01.789383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad6f3e-4ac1-446b-b9d2-a8ecb853c4d5 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:01.789485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeb4aa-2343-49c7-9e0a-f298aa205280 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:01.789697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeb4aa-2343-49c7-9e0a-f298aa205280 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:01.790548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eeb4aa-2343-49c7-9e0a-f298aa205280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:01.792206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eeb4aa-2343-49c7-9e0a-f298aa205280 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:01.795585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae773bf-eea4-47a2-8463-2eee87314b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:01.796448 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae773bf-eea4-47a2-8463-2eee87314b17 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:01.796811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae773bf-eea4-47a2-8463-2eee87314b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:01.797165 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae773bf-eea4-47a2-8463-2eee87314b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:01.798049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae773bf-eea4-47a2-8463-2eee87314b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:01.800525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae773bf-eea4-47a2-8463-2eee87314b17 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:01.815869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 859/1718] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:01.824185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae773bf-eea4-47a2-8463-2eee87314b17 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:01.839388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 860/1719] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:01.844255 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeb4aa-2343-49c7-9e0a-f298aa205280 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:01.851443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae773bf-eea4-47a2-8463-2eee87314b17 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:01.851572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e7eeb4aa-2343-49c7-9e0a-f298aa205280 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:01.865091 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc921b17-cf28-4d54-b7e6-1137582db0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:01.870235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc921b17-cf28-4d54-b7e6-1137582db0f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:01.871028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc921b17-cf28-4d54-b7e6-1137582db0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:01.872009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc921b17-cf28-4d54-b7e6-1137582db0f4 admin admin] PATH_INFO: `/v3/projects/0f8aa7d4bbd64c838eaad1a303e7a573/users/2cab6a8fe9c5400ba13bf02f723d77f2/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:01.875249 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc921b17-cf28-4d54-b7e6-1137582db0f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f8aa7d4bbd64c838eaad1a303e7a573, user_id=2cab6a8fe9c5400ba13bf02f723d77f2, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:01.884764 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc921b17-cf28-4d54-b7e6-1137582db0f4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:01.886491 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeb4aa-2343-49c7-9e0a-f298aa205280 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:01.897365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986f35c-06cf-496c-92ad-39a8270ec0a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:01.897365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986f35c-06cf-496c-92ad-39a8270ec0a6 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:01.897365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986f35c-06cf-496c-92ad-39a8270ec0a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:01.897365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986f35c-06cf-496c-92ad-39a8270ec0a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:01.921987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986f35c-06cf-496c-92ad-39a8270ec0a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:01.921987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986f35c-06cf-496c-92ad-39a8270ec0a6 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:01.941936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:01.943701 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 861/1720] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:01.944653 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986f35c-06cf-496c-92ad-39a8270ec0a6 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:01.950263 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 860/1721] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:01 2022] PUT /identity/v3/projects/0f8aa7d4bbd64c838eaad1a303e7a573/users/2cab6a8fe9c5400ba13bf02f723d77f2/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:01.965022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768fa66-f689-471c-aa71-8dd6146c2eca None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:01.965661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768fa66-f689-471c-aa71-8dd6146c2eca None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:01.966615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768fa66-f689-471c-aa71-8dd6146c2eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:01.978589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986f35c-06cf-496c-92ad-39a8270ec0a6 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:02.057403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:02.065833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-c768fa66-f689-471c-aa71-8dd6146c2eca None None] MFA Rules not processed for user `2cab6a8fe9c5400ba13bf02f723d77f2`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:02.067685 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c768fa66-f689-471c-aa71-8dd6146c2eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:02.112623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5732e-177b-432d-a29a-e0e0eec664a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:02.113561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5732e-177b-432d-a29a-e0e0eec664a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:02.113849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5732e-177b-432d-a29a-e0e0eec664a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:02.114131 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5732e-177b-432d-a29a-e0e0eec664a2 admin admin] PATH_INFO: `/v3/projects/125dec2e1f3646639722498943228e32/users/5d496976fb7e472eb2fec90414e2f683/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:02.149043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5732e-177b-432d-a29a-e0e0eec664a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=125dec2e1f3646639722498943228e32, user_id=5d496976fb7e472eb2fec90414e2f683, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:02.153815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d5732e-177b-432d-a29a-e0e0eec664a2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:02.187818 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 862/1722] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:01 2022] PUT /identity/v3/projects/125dec2e1f3646639722498943228e32/users/5d496976fb7e472eb2fec90414e2f683/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:02.203170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1802e9-1304-4a7e-990c-e105f1c7b86f None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:02.203498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1802e9-1304-4a7e-990c-e105f1c7b86f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:02.203848 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1802e9-1304-4a7e-990c-e105f1c7b86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:02.244501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 861/1723] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:01 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:02.260531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768fa66-f689-471c-aa71-8dd6146c2eca None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:02.266467 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c768fa66-f689-471c-aa71-8dd6146c2eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:02.283307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4c1802e9-1304-4a7e-990c-e105f1c7b86f None None] MFA Rules not processed for user `5d496976fb7e472eb2fec90414e2f683`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:02.284590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4c1802e9-1304-4a7e-990c-e105f1c7b86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:02.387188 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768fa66-f689-471c-aa71-8dd6146c2eca None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:02.438445 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 863/1724] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:02.442674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1802e9-1304-4a7e-990c-e105f1c7b86f None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:02.472387 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac451b7-145f-422b-9e9f-f4e575665511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:02.473285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac451b7-145f-422b-9e9f-f4e575665511 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:02.474221 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac451b7-145f-422b-9e9f-f4e575665511 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:02.474497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac451b7-145f-422b-9e9f-f4e575665511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:02.597289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:02.629701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac451b7-145f-422b-9e9f-f4e575665511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:02.631724 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac451b7-145f-422b-9e9f-f4e575665511 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:02.633551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1802e9-1304-4a7e-990c-e105f1c7b86f None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:02.646561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff6922-71bf-4e86-870a-d5d17a32c524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:02.647677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff6922-71bf-4e86-870a-d5d17a32c524 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:02.648278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff6922-71bf-4e86-870a-d5d17a32c524 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:02.648813 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff6922-71bf-4e86-870a-d5d17a32c524 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:02.649813 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff6922-71bf-4e86-870a-d5d17a32c524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:02.652028 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ff6922-71bf-4e86-870a-d5d17a32c524 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:02.671610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 862/1725] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:02 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:02.762974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 864/1726] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:02 2022] POST /identity/v3/projects => generated 371 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:02.766848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac451b7-145f-422b-9e9f-f4e575665511 service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:02.786397 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac451b7-145f-422b-9e9f-f4e575665511 service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:02.944426 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff6922-71bf-4e86-870a-d5d17a32c524 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:02.965982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f048368-5257-42ee-acdb-4b020f66c152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:02.966687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f048368-5257-42ee-acdb-4b020f66c152 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:02.966977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f048368-5257-42ee-acdb-4b020f66c152 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:02.967258 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f048368-5257-42ee-acdb-4b020f66c152 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:02.968088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f048368-5257-42ee-acdb-4b020f66c152 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:02.970030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f048368-5257-42ee-acdb-4b020f66c152 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:03.017369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 863/1727] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:02 2022] POST /identity/v3/users => generated 292 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:03.022469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f048368-5257-42ee-acdb-4b020f66c152 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:03.046087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:03.048334 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f048368-5257-42ee-acdb-4b020f66c152 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:03.060105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2f46b-66de-40d0-b70b-136b1b99bdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:03.061068 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2f46b-66de-40d0-b70b-136b1b99bdd6 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:03.061731 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2f46b-66de-40d0-b70b-136b1b99bdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:03.061731 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca2f46b-66de-40d0-b70b-136b1b99bdd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:03.062368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2f46b-66de-40d0-b70b-136b1b99bdd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:03.067779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca2f46b-66de-40d0-b70b-136b1b99bdd6 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:03.094962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 864/1728] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:03.098604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2f46b-66de-40d0-b70b-136b1b99bdd6 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:03.119230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca2f46b-66de-40d0-b70b-136b1b99bdd6 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:03.130491 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a8f47-dbb0-493a-88ad-0c6921435c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:03.132435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a8f47-dbb0-493a-88ad-0c6921435c7b admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:03.132730 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a8f47-dbb0-493a-88ad-0c6921435c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:03.133008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a8f47-dbb0-493a-88ad-0c6921435c7b admin admin] PATH_INFO: `/v3/projects/0702aa4b920445cd9fcdc74847a64718/users/dfc9954e49304747bf7bc654941676fe/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:03.177641 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a8f47-dbb0-493a-88ad-0c6921435c7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0702aa4b920445cd9fcdc74847a64718, user_id=dfc9954e49304747bf7bc654941676fe, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:03.181570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1a8f47-dbb0-493a-88ad-0c6921435c7b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:03.184383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff6922-71bf-4e86-870a-d5d17a32c524 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:03.218486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9710c-b51a-49e7-bf96-117ad5b02639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:03.220554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9710c-b51a-49e7-bf96-117ad5b02639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:03.221481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9710c-b51a-49e7-bf96-117ad5b02639 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:03.225206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9710c-b51a-49e7-bf96-117ad5b02639 admin admin] PATH_INFO: `/v3/users/de4e509aa41941248b8273ed06f7e23e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:03.227204 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9710c-b51a-49e7-bf96-117ad5b02639 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de4e509aa41941248b8273ed06f7e23e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:03.231979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 865/1729] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:03 2022] PUT /identity/v3/projects/0702aa4b920445cd9fcdc74847a64718/users/dfc9954e49304747bf7bc654941676fe/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:52:03.236274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a8f47-dbb0-493a-88ad-0c6921435c7b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:03.238283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9710c-b51a-49e7-bf96-117ad5b02639 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:03.427090 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b2d9710c-b51a-49e7-bf96-117ad5b02639 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de4e509aa41941248b8273ed06f7e23e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:03.435519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a8f47-dbb0-493a-88ad-0c6921435c7b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:03.455021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b2d9710c-b51a-49e7-bf96-117ad5b02639 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de4e509aa41941248b8273ed06f7e23e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:03.456898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b2d9710c-b51a-49e7-bf96-117ad5b02639 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de4e509aa41941248b8273ed06f7e23e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:03.464107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a471f24-df8a-43c9-b46c-83594f63f2ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:03.464912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a471f24-df8a-43c9-b46c-83594f63f2ec admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:03.465205 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a471f24-df8a-43c9-b46c-83594f63f2ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:03.467576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a471f24-df8a-43c9-b46c-83594f63f2ec admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:03.472292 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b2d9710c-b51a-49e7-bf96-117ad5b02639 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de4e509aa41941248b8273ed06f7e23e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:03.473153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b2d9710c-b51a-49e7-bf96-117ad5b02639 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de4e509aa41941248b8273ed06f7e23e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:03.473260 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a471f24-df8a-43c9-b46c-83594f63f2ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:03.480184 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a471f24-df8a-43c9-b46c-83594f63f2ec admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:03.503100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 865/1730] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:02 2022] DELETE /identity/v3/users/de4e509aa41941248b8273ed06f7e23e => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:03.508363 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c98b5-8549-4bc3-a386-8580e5373e34 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:03.508777 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c98b5-8549-4bc3-a386-8580e5373e34 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:03.509230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282c98b5-8549-4bc3-a386-8580e5373e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:03.517783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 866/1731] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:03.519698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a471f24-df8a-43c9-b46c-83594f63f2ec admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:03.528420 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4a471f24-df8a-43c9-b46c-83594f63f2ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:03.573557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-282c98b5-8549-4bc3-a386-8580e5373e34 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:03.576914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-282c98b5-8549-4bc3-a386-8580e5373e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:03.619947 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a471f24-df8a-43c9-b46c-83594f63f2ec admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:03.741887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:03.757430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 866/1732] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:03.781414 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c98b5-8549-4bc3-a386-8580e5373e34 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:03.783484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-282c98b5-8549-4bc3-a386-8580e5373e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:03.830751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8163dea8-0e50-4688-b6c7-d5ab6b89f407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:03.835197 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163dea8-0e50-4688-b6c7-d5ab6b89f407 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:03.835626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163dea8-0e50-4688-b6c7-d5ab6b89f407 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:03.837338 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163dea8-0e50-4688-b6c7-d5ab6b89f407 admin admin] PATH_INFO: `/v3/projects/0702aa4b920445cd9fcdc74847a64718/users/dfc9954e49304747bf7bc654941676fe/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:03.853181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163dea8-0e50-4688-b6c7-d5ab6b89f407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0702aa4b920445cd9fcdc74847a64718, user_id=dfc9954e49304747bf7bc654941676fe, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:03.859811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8163dea8-0e50-4688-b6c7-d5ab6b89f407 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:03.916106 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 867/1733] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:03 2022] PUT /identity/v3/projects/0702aa4b920445cd9fcdc74847a64718/users/dfc9954e49304747bf7bc654941676fe/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:03.918057 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f945c58-1f8e-4612-afd1-933dc829e877 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:03.918346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f945c58-1f8e-4612-afd1-933dc829e877 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:03.918624 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f945c58-1f8e-4612-afd1-933dc829e877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:03.979082 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-1f945c58-1f8e-4612-afd1-933dc829e877 None None] MFA Rules not processed for user `dfc9954e49304747bf7bc654941676fe`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:03.984897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1f945c58-1f8e-4612-afd1-933dc829e877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:03.991558 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282c98b5-8549-4bc3-a386-8580e5373e34 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:04.004681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:04.138611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26217091-7fcd-4f36-b90c-487f83c39d69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:04.140935 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26217091-7fcd-4f36-b90c-487f83c39d69 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:04.142553 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26217091-7fcd-4f36-b90c-487f83c39d69 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:04.143182 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26217091-7fcd-4f36-b90c-487f83c39d69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:04.193355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 868/1734] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:03 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:04.193355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f945c58-1f8e-4612-afd1-933dc829e877 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:04.193355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1f945c58-1f8e-4612-afd1-933dc829e877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:04.293077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26217091-7fcd-4f36-b90c-487f83c39d69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:04.296016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26217091-7fcd-4f36-b90c-487f83c39d69 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:04.316769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 867/1735] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:04.318301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26217091-7fcd-4f36-b90c-487f83c39d69 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:04.320231 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-26217091-7fcd-4f36-b90c-487f83c39d69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:04.409631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f945c58-1f8e-4612-afd1-933dc829e877 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:04.432940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cc545-6b1e-4f2c-b68f-a77bc7e371ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:04.435621 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cc545-6b1e-4f2c-b68f-a77bc7e371ff service placement] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:04.436234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cc545-6b1e-4f2c-b68f-a77bc7e371ff service placement] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:04.439985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658cc545-6b1e-4f2c-b68f-a77bc7e371ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:04.447851 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-658cc545-6b1e-4f2c-b68f-a77bc7e371ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:04.456451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26217091-7fcd-4f36-b90c-487f83c39d69 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:04.469221 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cc545-6b1e-4f2c-b68f-a77bc7e371ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:04.471553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658cc545-6b1e-4f2c-b68f-a77bc7e371ff service placement] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:04.472588 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb3c954-6410-4576-a120-523b6f637157 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:04.475123 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3c954-6410-4576-a120-523b6f637157 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:04.475722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3c954-6410-4576-a120-523b6f637157 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:04.476219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb3c954-6410-4576-a120-523b6f637157 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:04.488317 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 869/1736] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:04.491429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cc545-6b1e-4f2c-b68f-a77bc7e371ff service placement] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:04.493641 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-658cc545-6b1e-4f2c-b68f-a77bc7e371ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:04.508057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3c954-6410-4576-a120-523b6f637157 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:04.511690 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb3c954-6410-4576-a120-523b6f637157 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:04.512678 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658cc545-6b1e-4f2c-b68f-a77bc7e371ff service placement] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:04.519181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:04.520676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:04.520676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:04.521104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] PATH_INFO: `/v3/projects/b021b9c607ff4edfb338294a1faee71f` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:04.523720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b021b9c607ff4edfb338294a1faee71f)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:04.525407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:04.546087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 868/1737] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:04 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:04.550135 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cc47b-d297-4dba-8d46-905e00c8e9cb None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:04.550878 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cc47b-d297-4dba-8d46-905e00c8e9cb None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:04.551596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79cc47b-d297-4dba-8d46-905e00c8e9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:04.618466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-e79cc47b-d297-4dba-8d46-905e00c8e9cb None None] MFA Rules not processed for user `c4178d34dc114f46af725c9dc362d224`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:04.620505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e79cc47b-d297-4dba-8d46-905e00c8e9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:04.723133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:04.730841 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b021b9c607ff4edfb338294a1faee71f', 'request_id': 'req-5c90140f-15c2-4931-b1c5-3ccf242a219f'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:04.749494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b021b9c607ff4edfb338294a1faee71f', 'request_id': 'req-5c90140f-15c2-4931-b1c5-3ccf242a219f'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:04.760406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b021b9c607ff4edfb338294a1faee71f', 'request_id': 'req-5c90140f-15c2-4931-b1c5-3ccf242a219f'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:04.787816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] The token cache is being invalidate because project b021b9c607ff4edfb338294a1faee71f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:04.788406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:04.789390 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:04.795029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 870/1738] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:04 2022] DELETE /identity/v3/projects/b021b9c607ff4edfb338294a1faee71f => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:04.797265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:04.799463 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:04.818030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:04.878393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 869/1739] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:04.888586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9878f-2847-4cb5-8b84-fff57adea8e5 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:04.888880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9878f-2847-4cb5-8b84-fff57adea8e5 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:04.889163 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9878f-2847-4cb5-8b84-fff57adea8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:04.930075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-b4d9878f-2847-4cb5-8b84-fff57adea8e5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:04.931310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b4d9878f-2847-4cb5-8b84-fff57adea8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:05.009791 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90140f-15c2-4931-b1c5-3ccf242a219f admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:05.015594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:05.016508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:05.016804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:05.017131 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] PATH_INFO: `/v3/projects/47e3c3d4078b432cb23ba7b22add29eb` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:05.024179 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47e3c3d4078b432cb23ba7b22add29eb)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:05.026346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:05.129862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47e3c3d4078b432cb23ba7b22add29eb', 'request_id': 'req-d5cb8423-020f-476d-9232-b1a59dec2d20'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:05.142123 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47e3c3d4078b432cb23ba7b22add29eb', 'request_id': 'req-d5cb8423-020f-476d-9232-b1a59dec2d20'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:05.151591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47e3c3d4078b432cb23ba7b22add29eb', 'request_id': 'req-d5cb8423-020f-476d-9232-b1a59dec2d20'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:05.169702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] The token cache is being invalidate because project 47e3c3d4078b432cb23ba7b22add29eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:05.170090 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:05.170650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:05.172366 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 871/1741] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:04 2022] DELETE /identity/v3/projects/47e3c3d4078b432cb23ba7b22add29eb => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:05.173903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:05.174017 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 870/1741] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:05.175946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:05.179509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38907ff-e430-4f3d-97d9-b1d6c5ea4ec5 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:05.181322 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38907ff-e430-4f3d-97d9-b1d6c5ea4ec5 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:05.181634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38907ff-e430-4f3d-97d9-b1d6c5ea4ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:05.252399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-f38907ff-e430-4f3d-97d9-b1d6c5ea4ec5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:05.253440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f38907ff-e430-4f3d-97d9-b1d6c5ea4ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:05.398043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:05.413317 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb8423-020f-476d-9232-b1a59dec2d20 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:05.433938 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b7dee-5b18-443f-88d0-4805e2e603da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:05.435677 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b7dee-5b18-443f-88d0-4805e2e603da service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:05.436788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b7dee-5b18-443f-88d0-4805e2e603da service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:05.437081 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b7dee-5b18-443f-88d0-4805e2e603da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:05.447355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ae9b7dee-5b18-443f-88d0-4805e2e603da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:05.448873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 871/1742] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:05.449867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38907ff-e430-4f3d-97d9-b1d6c5ea4ec5 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:05.452543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f38907ff-e430-4f3d-97d9-b1d6c5ea4ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:05.480479 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38907ff-e430-4f3d-97d9-b1d6c5ea4ec5 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:05.512623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6561118-c279-4c40-a69d-bbbb77c9b6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:05.517729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6561118-c279-4c40-a69d-bbbb77c9b6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:05.517729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6561118-c279-4c40-a69d-bbbb77c9b6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:05.517729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6561118-c279-4c40-a69d-bbbb77c9b6b1 admin admin] PATH_INFO: `/v3/users/bc43823b53d143fb9a4f7db43b7c1fea` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:05.524741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6561118-c279-4c40-a69d-bbbb77c9b6b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc43823b53d143fb9a4f7db43b7c1fea)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:05.526656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6561118-c279-4c40-a69d-bbbb77c9b6b1 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:05.664224 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b7dee-5b18-443f-88d0-4805e2e603da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:05.667024 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b7dee-5b18-443f-88d0-4805e2e603da service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:05.680614 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 872/1743] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:05.682911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b7dee-5b18-443f-88d0-4805e2e603da service nova] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:05.701102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b7dee-5b18-443f-88d0-4805e2e603da service nova] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:05.702926 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f6561118-c279-4c40-a69d-bbbb77c9b6b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc43823b53d143fb9a4f7db43b7c1fea'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:05.718118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e8a06-196c-4a31-b534-abadd32960e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:05.719045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e8a06-196c-4a31-b534-abadd32960e6 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:05.719451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e8a06-196c-4a31-b534-abadd32960e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:05.720104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e8a06-196c-4a31-b534-abadd32960e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:05.721600 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e8a06-196c-4a31-b534-abadd32960e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:52:05.723122 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e8a06-196c-4a31-b534-abadd32960e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:05.726227 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e8a06-196c-4a31-b534-abadd32960e6 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:05.737989 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f6561118-c279-4c40-a69d-bbbb77c9b6b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc43823b53d143fb9a4f7db43b7c1fea'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:05.738969 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f6561118-c279-4c40-a69d-bbbb77c9b6b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc43823b53d143fb9a4f7db43b7c1fea'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:05.764693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f6561118-c279-4c40-a69d-bbbb77c9b6b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc43823b53d143fb9a4f7db43b7c1fea'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:05.765828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f6561118-c279-4c40-a69d-bbbb77c9b6b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc43823b53d143fb9a4f7db43b7c1fea'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:05.767547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 873/1744] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:52:05.769767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e8a06-196c-4a31-b534-abadd32960e6 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:05.782777 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4d5e8a06-196c-4a31-b534-abadd32960e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:05.816050 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 872/1745] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:05 2022] DELETE /identity/v3/users/bc43823b53d143fb9a4f7db43b7c1fea => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:05.820527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6561118-c279-4c40-a69d-bbbb77c9b6b1 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:05.823843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e8a06-196c-4a31-b534-abadd32960e6 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:05.824844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f6561118-c279-4c40-a69d-bbbb77c9b6b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:05.840504 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:06.018509 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71306358-d44a-4f44-9987-5381bd9daf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:06.020551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71306358-d44a-4f44-9987-5381bd9daf98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:06.020783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71306358-d44a-4f44-9987-5381bd9daf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:06.022270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71306358-d44a-4f44-9987-5381bd9daf98 admin admin] PATH_INFO: `/v3/users/5d496976fb7e472eb2fec90414e2f683` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:06.024879 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71306358-d44a-4f44-9987-5381bd9daf98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d496976fb7e472eb2fec90414e2f683)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:06.030452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71306358-d44a-4f44-9987-5381bd9daf98 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:06.062197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6561118-c279-4c40-a69d-bbbb77c9b6b1 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:06.088602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1aae4-01a0-448e-865e-49296db08a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:06.089310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1aae4-01a0-448e-865e-49296db08a4d admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:06.089597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1aae4-01a0-448e-865e-49296db08a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:06.089880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1aae4-01a0-448e-865e-49296db08a4d admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:06.090592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1aae4-01a0-448e-865e-49296db08a4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:06.138930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1aae4-01a0-448e-865e-49296db08a4d admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:06.225733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 873/1746] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:05 2022] POST /identity/v3/projects => generated 357 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:06.226782 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1aae4-01a0-448e-865e-49296db08a4d admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:06.233975 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-71306358-d44a-4f44-9987-5381bd9daf98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d496976fb7e472eb2fec90414e2f683'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:06.234293 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7cf1aae4-01a0-448e-865e-49296db08a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:06.291496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-71306358-d44a-4f44-9987-5381bd9daf98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d496976fb7e472eb2fec90414e2f683'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:06.311099 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-71306358-d44a-4f44-9987-5381bd9daf98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d496976fb7e472eb2fec90414e2f683'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:06.311812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-71306358-d44a-4f44-9987-5381bd9daf98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d496976fb7e472eb2fec90414e2f683'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:06.331701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-71306358-d44a-4f44-9987-5381bd9daf98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d496976fb7e472eb2fec90414e2f683'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:06.354568 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 874/1747] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:05 2022] DELETE /identity/v3/users/5d496976fb7e472eb2fec90414e2f683 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:06.356763 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71306358-d44a-4f44-9987-5381bd9daf98 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:06.361049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-71306358-d44a-4f44-9987-5381bd9daf98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:06.388971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:06.546343 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71306358-d44a-4f44-9987-5381bd9daf98 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:06.570661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b4d08b-fbfa-4244-b77a-0e8f750234e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:06.571372 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4d08b-fbfa-4244-b77a-0e8f750234e6 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:06.571660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4d08b-fbfa-4244-b77a-0e8f750234e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:06.572015 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b4d08b-fbfa-4244-b77a-0e8f750234e6 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:06.572728 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4d08b-fbfa-4244-b77a-0e8f750234e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:06.574740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b4d08b-fbfa-4244-b77a-0e8f750234e6 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:06.575272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1aae4-01a0-448e-865e-49296db08a4d admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:06.582316 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:06.584069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:06.584069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:06.584284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] PATH_INFO: `/v3/projects/27955b7afd6f493382198cdfc7bcb681` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:06.585905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] RBAC: Authorizing `identity:delete_project(project_id=27955b7afd6f493382198cdfc7bcb681)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:06.587957 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:06.623015 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 875/1748] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:06 2022] POST /identity/v3/users => generated 279 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:06.623991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b67d5-18d2-4d30-b4d6-41e8eac5fc33 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:06.624290 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b67d5-18d2-4d30-b4d6-41e8eac5fc33 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:06.624573 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b67d5-18d2-4d30-b4d6-41e8eac5fc33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:06.669026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-176b67d5-18d2-4d30-b4d6-41e8eac5fc33 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:06.671679 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-176b67d5-18d2-4d30-b4d6-41e8eac5fc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:06.707416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27955b7afd6f493382198cdfc7bcb681', 'request_id': 'req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:06.725807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27955b7afd6f493382198cdfc7bcb681', 'request_id': 'req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:06.751057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27955b7afd6f493382198cdfc7bcb681', 'request_id': 'req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:06.774463 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] The token cache is being invalidate because project 27955b7afd6f493382198cdfc7bcb681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:06.774767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:06.775531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:06.778441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 874/1749] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:06 2022] DELETE /identity/v3/projects/27955b7afd6f493382198cdfc7bcb681 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:06.779593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:06.785828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:06.862560 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 876/1750] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:06.870356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:06.871531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b67d5-18d2-4d30-b4d6-41e8eac5fc33 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:06.879205 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-176b67d5-18d2-4d30-b4d6-41e8eac5fc33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:06.939489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b67d5-18d2-4d30-b4d6-41e8eac5fc33 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:06.957015 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bf960-f009-4a0e-bd78-fc3d691980bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:06.957919 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bf960-f009-4a0e-bd78-fc3d691980bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:06.958204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bf960-f009-4a0e-bd78-fc3d691980bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:06.964490 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bf960-f009-4a0e-bd78-fc3d691980bf admin admin] PATH_INFO: `/v3/users/2cab6a8fe9c5400ba13bf02f723d77f2` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:06.973819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9bf960-f009-4a0e-bd78-fc3d691980bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cab6a8fe9c5400ba13bf02f723d77f2)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:06.980253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9bf960-f009-4a0e-bd78-fc3d691980bf admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:07.016687 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c85fc-9b7b-41fd-97e6-1f598b76780d admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:07.042803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc41b2f-5535-40d2-87d9-4548e23155f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:07.043522 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41b2f-5535-40d2-87d9-4548e23155f2 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:07.050865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41b2f-5535-40d2-87d9-4548e23155f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:07.051229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc41b2f-5535-40d2-87d9-4548e23155f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:07.052109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc41b2f-5535-40d2-87d9-4548e23155f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:07.057165 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc41b2f-5535-40d2-87d9-4548e23155f2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:07.143678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 875/1751] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:06 2022] GET /identity/v3/roles => generated 1420 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:07.145112 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc41b2f-5535-40d2-87d9-4548e23155f2 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:07.149674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9fc41b2f-5535-40d2-87d9-4548e23155f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:07.159673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-2e9bf960-f009-4a0e-bd78-fc3d691980bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cab6a8fe9c5400ba13bf02f723d77f2'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:07.196049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-2e9bf960-f009-4a0e-bd78-fc3d691980bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cab6a8fe9c5400ba13bf02f723d77f2'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:07.210142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-2e9bf960-f009-4a0e-bd78-fc3d691980bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cab6a8fe9c5400ba13bf02f723d77f2'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:07.210859 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-2e9bf960-f009-4a0e-bd78-fc3d691980bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cab6a8fe9c5400ba13bf02f723d77f2'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:07.223839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-2e9bf960-f009-4a0e-bd78-fc3d691980bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cab6a8fe9c5400ba13bf02f723d77f2'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:07.249305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 877/1752] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:06 2022] DELETE /identity/v3/users/2cab6a8fe9c5400ba13bf02f723d77f2 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:07.253539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bf960-f009-4a0e-bd78-fc3d691980bf admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:07.254890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-2e9bf960-f009-4a0e-bd78-fc3d691980bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:07.281153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:07.370377 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc41b2f-5535-40d2-87d9-4548e23155f2 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:07.449063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bf960-f009-4a0e-bd78-fc3d691980bf admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:07.462000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b393739f-ed57-4e5e-bacc-18e3a5e72c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:07.462816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393739f-ed57-4e5e-bacc-18e3a5e72c47 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:07.463109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393739f-ed57-4e5e-bacc-18e3a5e72c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:07.463438 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b393739f-ed57-4e5e-bacc-18e3a5e72c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:07.464214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393739f-ed57-4e5e-bacc-18e3a5e72c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:07.466352 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b393739f-ed57-4e5e-bacc-18e3a5e72c47 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:07.483899 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 878/1753] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:07 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:07.485881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72116926-ea86-4507-8e6d-f6ed3b04abc1 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:07.486357 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72116926-ea86-4507-8e6d-f6ed3b04abc1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:07.487529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72116926-ea86-4507-8e6d-f6ed3b04abc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:07.536325 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870a25f-496d-4a1e-8b50-932d871ce807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:07.537173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870a25f-496d-4a1e-8b50-932d871ce807 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:07.537290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870a25f-496d-4a1e-8b50-932d871ce807 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:07.537585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6870a25f-496d-4a1e-8b50-932d871ce807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:07.546467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6870a25f-496d-4a1e-8b50-932d871ce807 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:07.556942 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-72116926-ea86-4507-8e6d-f6ed3b04abc1 None None] MFA Rules not processed for user `84c38f6ff5af4ba79c80fb581baacb0b`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:07.559665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-72116926-ea86-4507-8e6d-f6ed3b04abc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:07.580702 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870a25f-496d-4a1e-8b50-932d871ce807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:07.598690 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6870a25f-496d-4a1e-8b50-932d871ce807 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:07.621913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 876/1754] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:07.623256 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870a25f-496d-4a1e-8b50-932d871ce807 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:07.640137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6870a25f-496d-4a1e-8b50-932d871ce807 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:07.650394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7320c065-5386-493b-a428-81c5a2544c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:07.651975 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320c065-5386-493b-a428-81c5a2544c75 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:07.652093 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320c065-5386-493b-a428-81c5a2544c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:07.652542 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7320c065-5386-493b-a428-81c5a2544c75 admin admin] PATH_INFO: `/v3/projects/8c27532b39e645cfbfe67d989eb5aade/users/c31dff105f89478194151a4450fde854/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:07.689469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320c065-5386-493b-a428-81c5a2544c75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c27532b39e645cfbfe67d989eb5aade, user_id=c31dff105f89478194151a4450fde854, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:07.692421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7320c065-5386-493b-a428-81c5a2544c75 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:07.736373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 877/1755] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:07 2022] PUT /identity/v3/projects/8c27532b39e645cfbfe67d989eb5aade/users/c31dff105f89478194151a4450fde854/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:07.750800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1de82-5abc-49f9-8355-bc2559d979b1 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:07.751337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1de82-5abc-49f9-8355-bc2559d979b1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:07.751693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1de82-5abc-49f9-8355-bc2559d979b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:07.828934 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-40d1de82-5abc-49f9-8355-bc2559d979b1 None None] MFA Rules not processed for user `c31dff105f89478194151a4450fde854`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:07.833485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-40d1de82-5abc-49f9-8355-bc2559d979b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:07.908044 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 879/1756] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:07.926073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72116926-ea86-4507-8e6d-f6ed3b04abc1 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:07.929516 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-72116926-ea86-4507-8e6d-f6ed3b04abc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:07.988032 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:08.067698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 878/1757] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:07 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:08.068957 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1de82-5abc-49f9-8355-bc2559d979b1 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:08.203006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72116926-ea86-4507-8e6d-f6ed3b04abc1 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:08.255506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d1de82-5abc-49f9-8355-bc2559d979b1 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:08.288763 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79b28c-d230-470e-9504-d5f20794b05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:08.290566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79b28c-d230-470e-9504-d5f20794b05a admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:08.301631 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79b28c-d230-470e-9504-d5f20794b05a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:08.302091 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d79b28c-d230-470e-9504-d5f20794b05a admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:08.309017 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79b28c-d230-470e-9504-d5f20794b05a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:08.316593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d79b28c-d230-470e-9504-d5f20794b05a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:08.335565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:08.418724 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c15140-03bf-4522-aa53-2b686d907b2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:08.419456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 879/1758] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:08 2022] POST /identity/v3/projects => generated 355 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:08.419772 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c15140-03bf-4522-aa53-2b686d907b2f service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:08.420291 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c15140-03bf-4522-aa53-2b686d907b2f service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:08.420745 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c15140-03bf-4522-aa53-2b686d907b2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:08.432011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79b28c-d230-470e-9504-d5f20794b05a admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:08.551387 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c15140-03bf-4522-aa53-2b686d907b2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:08.553300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c15140-03bf-4522-aa53-2b686d907b2f service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:08.580198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 880/1759] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 654 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:52:08.583177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a83dfe-169f-43c1-b392-e7f1a6b2f6c8 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:08.583580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a83dfe-169f-43c1-b392-e7f1a6b2f6c8 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:08.583995 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a83dfe-169f-43c1-b392-e7f1a6b2f6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:08.599814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d79b28c-d230-470e-9504-d5f20794b05a admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:08.617794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86a1f2-20b2-4188-9794-c6d41d96848f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:08.618671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86a1f2-20b2-4188-9794-c6d41d96848f admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:08.619104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86a1f2-20b2-4188-9794-c6d41d96848f admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:08.619546 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b86a1f2-20b2-4188-9794-c6d41d96848f admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:08.620440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86a1f2-20b2-4188-9794-c6d41d96848f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:08.622709 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b86a1f2-20b2-4188-9794-c6d41d96848f admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:08.672332 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 880/1760] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:08 2022] POST /identity/v3/users => generated 284 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:08.673520 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86a1f2-20b2-4188-9794-c6d41d96848f admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:08.682181 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0b86a1f2-20b2-4188-9794-c6d41d96848f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:08.698872 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-75a83dfe-169f-43c1-b392-e7f1a6b2f6c8 None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:08.703420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b86a1f2-20b2-4188-9794-c6d41d96848f admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:08.703609 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-75a83dfe-169f-43c1-b392-e7f1a6b2f6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:08.716964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:08.717674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:08.718101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:08.718246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] PATH_INFO: `/v3/projects/7c60a51fd76a4b16a9fba425c9daf02c` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:08.719567 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c60a51fd76a4b16a9fba425c9daf02c)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:08.721578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:08.852805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c60a51fd76a4b16a9fba425c9daf02c', 'request_id': 'req-ee02de38-a59e-4a34-872f-0ee26925c147'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:08.854912 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:08.864280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c60a51fd76a4b16a9fba425c9daf02c', 'request_id': 'req-ee02de38-a59e-4a34-872f-0ee26925c147'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:08.885831 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c60a51fd76a4b16a9fba425c9daf02c', 'request_id': 'req-ee02de38-a59e-4a34-872f-0ee26925c147'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:08.889806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 881/1761] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 17:52:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:08.891101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a83dfe-169f-43c1-b392-e7f1a6b2f6c8 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:08.908669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] The token cache is being invalidate because project 7c60a51fd76a4b16a9fba425c9daf02c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:08.909126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:08.910668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:08.913887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 881/1762] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:08 2022] DELETE /identity/v3/projects/7c60a51fd76a4b16a9fba425c9daf02c => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:08.917254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:08.917466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:08.921016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a83dfe-169f-43c1-b392-e7f1a6b2f6c8 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:09.036406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02de38-a59e-4a34-872f-0ee26925c147 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:09.054755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:09.056492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:09.056774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:09.057432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] PATH_INFO: `/v3/projects/0f8aa7d4bbd64c838eaad1a303e7a573` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:09.064102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f8aa7d4bbd64c838eaad1a303e7a573)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:09.066681 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:09.136855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:09.188197 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f05a4-cf00-4e1f-bdf1-638e5607fd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:09.189238 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f05a4-cf00-4e1f-bdf1-638e5607fd22 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:09.189877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f05a4-cf00-4e1f-bdf1-638e5607fd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:09.190309 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f05a4-cf00-4e1f-bdf1-638e5607fd22 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:09.191307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f05a4-cf00-4e1f-bdf1-638e5607fd22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:09.196202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f05a4-cf00-4e1f-bdf1-638e5607fd22 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:09.226642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 882/1763] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:08 2022] GET /identity/v3/roles => generated 1420 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:09.229026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a30384-2050-49d0-84e4-9e539610cc4a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:09.229454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a30384-2050-49d0-84e4-9e539610cc4a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:09.230260 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a30384-2050-49d0-84e4-9e539610cc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:09.245425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f8aa7d4bbd64c838eaad1a303e7a573', 'request_id': 'req-ed50d3e6-d202-4343-820b-cf5d83643127'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:09.275251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f8aa7d4bbd64c838eaad1a303e7a573', 'request_id': 'req-ed50d3e6-d202-4343-820b-cf5d83643127'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:09.296474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f8aa7d4bbd64c838eaad1a303e7a573', 'request_id': 'req-ed50d3e6-d202-4343-820b-cf5d83643127'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:09.304707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f6a30384-2050-49d0-84e4-9e539610cc4a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:09.306017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f6a30384-2050-49d0-84e4-9e539610cc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:09.327122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] The token cache is being invalidate because project 0f8aa7d4bbd64c838eaad1a303e7a573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:09.327735 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:09.329551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:09.332826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 882/1764] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:08 2022] DELETE /identity/v3/projects/0f8aa7d4bbd64c838eaad1a303e7a573 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:09.335306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:09.341809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:09.451909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 883/1765] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:09.451909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a30384-2050-49d0-84e4-9e539610cc4a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:09.452962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f6a30384-2050-49d0-84e4-9e539610cc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:09.492029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50d3e6-d202-4343-820b-cf5d83643127 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:09.502320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:09.504407 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:09.505220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:09.505607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] PATH_INFO: `/v3/projects/125dec2e1f3646639722498943228e32` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:09.510139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=125dec2e1f3646639722498943228e32)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:09.514725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:09.531963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:09.662187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '125dec2e1f3646639722498943228e32', 'request_id': 'req-171ff976-251f-447b-810b-691d3a5777c7'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:09.689838 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '125dec2e1f3646639722498943228e32', 'request_id': 'req-171ff976-251f-447b-810b-691d3a5777c7'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:09.699081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '125dec2e1f3646639722498943228e32', 'request_id': 'req-171ff976-251f-447b-810b-691d3a5777c7'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:09.701540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a30384-2050-49d0-84e4-9e539610cc4a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:09.729331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] The token cache is being invalidate because project 125dec2e1f3646639722498943228e32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:09.729638 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:09.730368 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:09.735880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 883/1766] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:09 2022] DELETE /identity/v3/projects/125dec2e1f3646639722498943228e32 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:09.737055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:09.738899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:09.877166 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ff976-251f-447b-810b-691d3a5777c7 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:09.880637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:09.894268 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd97e5-33a8-431e-bc39-6683544dbe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:09.895206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd97e5-33a8-431e-bc39-6683544dbe64 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:09.896681 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975e635-de83-4864-a584-78dd739e71f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:09.897498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd97e5-33a8-431e-bc39-6683544dbe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:09.897793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd97e5-33a8-431e-bc39-6683544dbe64 admin admin] PATH_INFO: `/v3/projects/be858e8d9e8d4164bfae8802b5e4d199/users/9cdf7f997ff34a22ac1029c6ece4e419/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:09.898789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975e635-de83-4864-a584-78dd739e71f9 service swift] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:09.899179 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975e635-de83-4864-a584-78dd739e71f9 service swift] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:09.900166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975e635-de83-4864-a584-78dd739e71f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:09.904568 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e975e635-de83-4864-a584-78dd739e71f9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:09.941871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd97e5-33a8-431e-bc39-6683544dbe64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be858e8d9e8d4164bfae8802b5e4d199, user_id=9cdf7f997ff34a22ac1029c6ece4e419, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:09.944979 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd97e5-33a8-431e-bc39-6683544dbe64 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:09.975868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 884/1767] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:09 2022] PUT /identity/v3/projects/be858e8d9e8d4164bfae8802b5e4d199/users/9cdf7f997ff34a22ac1029c6ece4e419/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:09.977719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7200487-0c25-4d72-aef9-393675a0d715 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:09.980109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7200487-0c25-4d72-aef9-393675a0d715 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:09.980370 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7200487-0c25-4d72-aef9-393675a0d715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:10.025504 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-a7200487-0c25-4d72-aef9-393675a0d715 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:10.029442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a7200487-0c25-4d72-aef9-393675a0d715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:10.051379 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975e635-de83-4864-a584-78dd739e71f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:10.053581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975e635-de83-4864-a584-78dd739e71f9 service swift] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:10.073648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 884/1768] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 17:52:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 17:52:10.075066 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ace37-4609-4ff3-8e62-a0c6392365bf None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:10.075356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ace37-4609-4ff3-8e62-a0c6392365bf None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:10.075632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54ace37-4609-4ff3-8e62-a0c6392365bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:10.130769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b54ace37-4609-4ff3-8e62-a0c6392365bf None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:10.133785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b54ace37-4609-4ff3-8e62-a0c6392365bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:10.182790 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 885/1769] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:10.185318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7200487-0c25-4d72-aef9-393675a0d715 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:10.187043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a7200487-0c25-4d72-aef9-393675a0d715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:10.211433 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7200487-0c25-4d72-aef9-393675a0d715 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:10.223447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848874e-3156-4206-a289-b9996ac2809c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:10.224725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848874e-3156-4206-a289-b9996ac2809c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:10.225015 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848874e-3156-4206-a289-b9996ac2809c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:10.225379 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848874e-3156-4206-a289-b9996ac2809c admin admin] PATH_INFO: `/v3/users/dfc9954e49304747bf7bc654941676fe` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:10.228590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848874e-3156-4206-a289-b9996ac2809c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfc9954e49304747bf7bc654941676fe)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:10.230728 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848874e-3156-4206-a289-b9996ac2809c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:10.260832 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:10.291198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 885/1770] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:10.292782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ace37-4609-4ff3-8e62-a0c6392365bf None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:10.334229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-9848874e-3156-4206-a289-b9996ac2809c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfc9954e49304747bf7bc654941676fe'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:10.337412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54ace37-4609-4ff3-8e62-a0c6392365bf None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:10.358087 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ef1fa-7441-4dbc-beb8-70c2b142def9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:10.358808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ef1fa-7441-4dbc-beb8-70c2b142def9 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:10.359101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ef1fa-7441-4dbc-beb8-70c2b142def9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:10.359386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ef1fa-7441-4dbc-beb8-70c2b142def9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:10.363578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ef1fa-7441-4dbc-beb8-70c2b142def9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:10.363888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-9848874e-3156-4206-a289-b9996ac2809c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfc9954e49304747bf7bc654941676fe'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:10.364860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-9848874e-3156-4206-a289-b9996ac2809c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfc9954e49304747bf7bc654941676fe'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:10.369647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ef1fa-7441-4dbc-beb8-70c2b142def9 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:10.382475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-9848874e-3156-4206-a289-b9996ac2809c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfc9954e49304747bf7bc654941676fe'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:10.383472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-9848874e-3156-4206-a289-b9996ac2809c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfc9954e49304747bf7bc654941676fe'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:10.409470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 886/1771] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:10 2022] DELETE /identity/v3/users/dfc9954e49304747bf7bc654941676fe => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:10.409718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 886/1772] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:10.411139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ef1fa-7441-4dbc-beb8-70c2b142def9 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:10.416845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e432c3-bd07-4492-b764-2b00af00e705 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:10.417269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e432c3-bd07-4492-b764-2b00af00e705 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:10.417574 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e432c3-bd07-4492-b764-2b00af00e705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:10.422812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-cc0ef1fa-7441-4dbc-beb8-70c2b142def9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:10.461599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-d7e432c3-bd07-4492-b764-2b00af00e705 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:10.463705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d7e432c3-bd07-4492-b764-2b00af00e705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:10.468255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ef1fa-7441-4dbc-beb8-70c2b142def9 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:10.659943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd438b11-6add-43c6-b74f-4791305fee2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:10.661014 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 887/1773] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:10.661648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd438b11-6add-43c6-b74f-4791305fee2b admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:10.661941 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd438b11-6add-43c6-b74f-4791305fee2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:10.662218 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd438b11-6add-43c6-b74f-4791305fee2b admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:10.663012 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd438b11-6add-43c6-b74f-4791305fee2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:52:10.664093 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e432c3-bd07-4492-b764-2b00af00e705 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:10.668027 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd438b11-6add-43c6-b74f-4791305fee2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:10.670413 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d7e432c3-bd07-4492-b764-2b00af00e705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:10.677893 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd438b11-6add-43c6-b74f-4791305fee2b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:10.716316 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e432c3-bd07-4492-b764-2b00af00e705 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:10.724739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 887/1774] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:52:10.726495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd438b11-6add-43c6-b74f-4791305fee2b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:10.732751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083a5eb-2881-4cfd-be56-baa42317b87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:10.732751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083a5eb-2881-4cfd-be56-baa42317b87d admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:10.732751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083a5eb-2881-4cfd-be56-baa42317b87d admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:10.733462 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c083a5eb-2881-4cfd-be56-baa42317b87d admin admin] PATH_INFO: `/v3/projects/be858e8d9e8d4164bfae8802b5e4d199/users/9cdf7f997ff34a22ac1029c6ece4e419/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:10.739018 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083a5eb-2881-4cfd-be56-baa42317b87d admin admin] RBAC: Authorizing `identity:create_grant(project_id=be858e8d9e8d4164bfae8802b5e4d199, user_id=9cdf7f997ff34a22ac1029c6ece4e419, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:10.744979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd438b11-6add-43c6-b74f-4791305fee2b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:10.746071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c083a5eb-2881-4cfd-be56-baa42317b87d admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:10.759584 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7293e-7304-479b-8b61-d5346287d541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:10.761566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7293e-7304-479b-8b61-d5346287d541 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:10.761887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7293e-7304-479b-8b61-d5346287d541 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:10.762179 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7293e-7304-479b-8b61-d5346287d541 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:10.763304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7293e-7304-479b-8b61-d5346287d541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:52:10.763670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7293e-7304-479b-8b61-d5346287d541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:10.765597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7293e-7304-479b-8b61-d5346287d541 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:10.793365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 888/1775] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:10 2022] PUT /identity/v3/projects/be858e8d9e8d4164bfae8802b5e4d199/users/9cdf7f997ff34a22ac1029c6ece4e419/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:10.810119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 888/1776] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:52:10.817477 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889c622-7c9a-4a43-b2d8-39327510efe9 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:10.817702 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083a5eb-2881-4cfd-be56-baa42317b87d admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:10.817988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889c622-7c9a-4a43-b2d8-39327510efe9 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:10.818329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889c622-7c9a-4a43-b2d8-39327510efe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:10.910108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:10.922382 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-8889c622-7c9a-4a43-b2d8-39327510efe9 None None] MFA Rules not processed for user `9cdf7f997ff34a22ac1029c6ece4e419`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:10.927500 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8889c622-7c9a-4a43-b2d8-39327510efe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:11.058894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c083a5eb-2881-4cfd-be56-baa42317b87d admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:11.075526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bd57d-1c7f-4656-b118-f25dcd4c3743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:11.076126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bd57d-1c7f-4656-b118-f25dcd4c3743 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:11.076518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bd57d-1c7f-4656-b118-f25dcd4c3743 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:11.076668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7bd57d-1c7f-4656-b118-f25dcd4c3743 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:11.077367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7bd57d-1c7f-4656-b118-f25dcd4c3743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:11.079329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7bd57d-1c7f-4656-b118-f25dcd4c3743 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:11.149814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 889/1777] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:10 2022] POST /identity/v3/projects => generated 399 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:11.149814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bd57d-1c7f-4656-b118-f25dcd4c3743 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:11.162826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 889/1778] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:10 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:11.162826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889c622-7c9a-4a43-b2d8-39327510efe9 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:11.217589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889c622-7c9a-4a43-b2d8-39327510efe9 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:11.221624 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:11.462523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7bd57d-1c7f-4656-b118-f25dcd4c3743 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:11.478122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcc2ef-24f7-4d68-b86a-410def00b593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:11.479140 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f6f56-6063-43dc-b7ff-fc5b10fa96da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:11.479862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f6f56-6063-43dc-b7ff-fc5b10fa96da admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:11.483073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcc2ef-24f7-4d68-b86a-410def00b593 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:11.483513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcc2ef-24f7-4d68-b86a-410def00b593 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:11.484042 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f6f56-6063-43dc-b7ff-fc5b10fa96da admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:11.484342 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f6f56-6063-43dc-b7ff-fc5b10fa96da admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:11.484817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcc2ef-24f7-4d68-b86a-410def00b593 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:11.485147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f6f56-6063-43dc-b7ff-fc5b10fa96da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:11.487437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bcc2ef-24f7-4d68-b86a-410def00b593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:11.493937 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bcc2ef-24f7-4d68-b86a-410def00b593 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:11.494389 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f6f56-6063-43dc-b7ff-fc5b10fa96da admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:11.585307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 890/1779] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:11 2022] POST /identity/v3/projects => generated 371 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:11.585307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcc2ef-24f7-4d68-b86a-410def00b593 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:11.590958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f2bcc2ef-24f7-4d68-b86a-410def00b593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:11.594874 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 890/1780] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:11 2022] POST /identity/v3/users => generated 300 bytes in 435 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:11.605455 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfdc38-aee6-42c6-96d9-ac4627c59fa1 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:11.613761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfdc38-aee6-42c6-96d9-ac4627c59fa1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:11.614253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfdc38-aee6-42c6-96d9-ac4627c59fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:11.721137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f9cfdc38-aee6-42c6-96d9-ac4627c59fa1 None None] MFA Rules not processed for user `52c794c11ab74f9ca9bad75857043a62`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:11.723333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f9cfdc38-aee6-42c6-96d9-ac4627c59fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:11.763252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcc2ef-24f7-4d68-b86a-410def00b593 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:11.788876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c4478-f747-4ad7-acf0-a7753e50dbff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:11.792947 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c4478-f747-4ad7-acf0-a7753e50dbff service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:11.793383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c4478-f747-4ad7-acf0-a7753e50dbff service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:11.793775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c4478-f747-4ad7-acf0-a7753e50dbff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:12.018008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 891/1781] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:12.022880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfdc38-aee6-42c6-96d9-ac4627c59fa1 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:12.067770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c4478-f747-4ad7-acf0-a7753e50dbff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:12.086105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c4478-f747-4ad7-acf0-a7753e50dbff service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:12.119561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:12.157078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 891/1782] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:11 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:12.172849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c4478-f747-4ad7-acf0-a7753e50dbff service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:12.296484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:12.352488 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c4478-f747-4ad7-acf0-a7753e50dbff service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:12.352824 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfdc38-aee6-42c6-96d9-ac4627c59fa1 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:12.358304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc7750-67c8-4f75-9027-08e5db246a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:12.360500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc7750-67c8-4f75-9027-08e5db246a88 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:12.362068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc7750-67c8-4f75-9027-08e5db246a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:12.362787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc7750-67c8-4f75-9027-08e5db246a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:12.364255 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc7750-67c8-4f75-9027-08e5db246a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:12.367067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc7750-67c8-4f75-9027-08e5db246a88 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:12.384104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f17f62-ae89-4db2-b169-083008cccead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:12.385206 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f17f62-ae89-4db2-b169-083008cccead admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:12.385672 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f17f62-ae89-4db2-b169-083008cccead admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:12.385957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f17f62-ae89-4db2-b169-083008cccead admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:12.386712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f17f62-ae89-4db2-b169-083008cccead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:12.389015 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f17f62-ae89-4db2-b169-083008cccead admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:12.405540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 892/1783] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:12.411577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d1745-9d9e-4a64-b324-8e1b938ffe97 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:12.412063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d1745-9d9e-4a64-b324-8e1b938ffe97 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:12.414161 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d1745-9d9e-4a64-b324-8e1b938ffe97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:12.465569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 892/1784] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:12 2022] POST /identity/v3/users => generated 286 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:12.469424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f17f62-ae89-4db2-b169-083008cccead admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:12.486880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-e85d1745-9d9e-4a64-b324-8e1b938ffe97 None None] MFA Rules not processed for user `52c794c11ab74f9ca9bad75857043a62`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:12.492699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e85d1745-9d9e-4a64-b324-8e1b938ffe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:12.511866 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f17f62-ae89-4db2-b169-083008cccead admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:12.523379 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 893/1785] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:12.529717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d1745-9d9e-4a64-b324-8e1b938ffe97 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:12.533550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e85d1745-9d9e-4a64-b324-8e1b938ffe97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:12.537008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdd035-25c0-4b30-aa68-9f84cc021395 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:12.538329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdd035-25c0-4b30-aa68-9f84cc021395 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:12.539012 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdd035-25c0-4b30-aa68-9f84cc021395 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:12.539546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cdd035-25c0-4b30-aa68-9f84cc021395 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:12.553098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a1cdd035-25c0-4b30-aa68-9f84cc021395 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:12.586843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d1745-9d9e-4a64-b324-8e1b938ffe97 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:12.598051 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:12.600841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:12.604243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:12.604667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] PATH_INFO: `/v3/projects/0702aa4b920445cd9fcdc74847a64718` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:12.606217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0702aa4b920445cd9fcdc74847a64718)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:12.610492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:12.722267 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0702aa4b920445cd9fcdc74847a64718', 'request_id': 'req-1392be77-a2cd-401a-881e-b3ebcf1120fc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:12.729990 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdd035-25c0-4b30-aa68-9f84cc021395 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:12.732158 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cdd035-25c0-4b30-aa68-9f84cc021395 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:12.739254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0702aa4b920445cd9fcdc74847a64718', 'request_id': 'req-1392be77-a2cd-401a-881e-b3ebcf1120fc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:12.748494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0702aa4b920445cd9fcdc74847a64718', 'request_id': 'req-1392be77-a2cd-401a-881e-b3ebcf1120fc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:12.783806 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] The token cache is being invalidate because project 0702aa4b920445cd9fcdc74847a64718 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:12.784452 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:12.784806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 893/1786] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:12 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:12.785846 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdd035-25c0-4b30-aa68-9f84cc021395 service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:12.787771 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:12.806126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 894/1787] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:12 2022] DELETE /identity/v3/projects/0702aa4b920445cd9fcdc74847a64718 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:12.806126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:12.807505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:12.844758 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cdd035-25c0-4b30-aa68-9f84cc021395 service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:13.060819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392be77-a2cd-401a-881e-b3ebcf1120fc admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:13.080479 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1ae78-c652-40a0-98e5-1dce79b360ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:13.083211 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1ae78-c652-40a0-98e5-1dce79b360ab admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:13.083669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1ae78-c652-40a0-98e5-1dce79b360ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:13.084665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc1ae78-c652-40a0-98e5-1dce79b360ab admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:13.086275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1ae78-c652-40a0-98e5-1dce79b360ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:13.088813 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc1ae78-c652-40a0-98e5-1dce79b360ab admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:13.109700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2150a4-fee5-4011-88df-e51b94a1aec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:13.110411 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2150a4-fee5-4011-88df-e51b94a1aec3 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:13.112858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2150a4-fee5-4011-88df-e51b94a1aec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:13.113391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2150a4-fee5-4011-88df-e51b94a1aec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:13.114333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2150a4-fee5-4011-88df-e51b94a1aec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:13.118132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 895/1788] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:13.118313 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2150a4-fee5-4011-88df-e51b94a1aec3 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:13.119318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1ae78-c652-40a0-98e5-1dce79b360ab admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:13.124588 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-bbc1ae78-c652-40a0-98e5-1dce79b360ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:13.164609 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 894/1789] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:12 2022] GET /identity/v3/roles => generated 1420 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:13.188776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2b828-abdd-4075-9915-254f3487d9c8 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:13.189334 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2b828-abdd-4075-9915-254f3487d9c8 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:13.189723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a2b828-abdd-4075-9915-254f3487d9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:13.277963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b5a2b828-abdd-4075-9915-254f3487d9c8 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:13.285389 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b5a2b828-abdd-4075-9915-254f3487d9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:13.288298 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:13.325836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 895/1790] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:13.326879 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2b828-abdd-4075-9915-254f3487d9c8 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:13.349891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a2b828-abdd-4075-9915-254f3487d9c8 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:13.374780 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc1ae78-c652-40a0-98e5-1dce79b360ab admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:13.379275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f94188-4c2c-45ff-8983-88db6d66fa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:13.380972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f94188-4c2c-45ff-8983-88db6d66fa57 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:13.382882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f94188-4c2c-45ff-8983-88db6d66fa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:13.384024 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f94188-4c2c-45ff-8983-88db6d66fa57 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:13.386272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f94188-4c2c-45ff-8983-88db6d66fa57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:13.391219 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f94188-4c2c-45ff-8983-88db6d66fa57 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:13.395990 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dc2dd-568e-4b85-99eb-d22bdfd6b3a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:13.396835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dc2dd-568e-4b85-99eb-d22bdfd6b3a1 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:13.398099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dc2dd-568e-4b85-99eb-d22bdfd6b3a1 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:13.399096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dc2dd-568e-4b85-99eb-d22bdfd6b3a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:13.411404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a36dc2dd-568e-4b85-99eb-d22bdfd6b3a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:13.435615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 896/1791] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:13 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:13.455778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f94188-4c2c-45ff-8983-88db6d66fa57 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:13.462247 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e8f94188-4c2c-45ff-8983-88db6d66fa57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:13.515887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f94188-4c2c-45ff-8983-88db6d66fa57 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:13.535862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69779e-fbbe-445a-945f-62241e9f3718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:13.541991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69779e-fbbe-445a-945f-62241e9f3718 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:13.542420 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69779e-fbbe-445a-945f-62241e9f3718 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:13.542709 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69779e-fbbe-445a-945f-62241e9f3718 admin admin] PATH_INFO: `/v3/projects/df34a1d1b06f46fba7c5a7963e5019a8/users/87b956accf5849a7a8f73de53064948e/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:13.609236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69779e-fbbe-445a-945f-62241e9f3718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df34a1d1b06f46fba7c5a7963e5019a8, user_id=87b956accf5849a7a8f73de53064948e, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:13.633274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69779e-fbbe-445a-945f-62241e9f3718 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:13.672589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:13.702895 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 897/1792] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:13 2022] PUT /identity/v3/projects/df34a1d1b06f46fba7c5a7963e5019a8/users/87b956accf5849a7a8f73de53064948e/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:13.704741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a6355-cbb2-483a-8012-96f15f27ff5d None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:13.707541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a6355-cbb2-483a-8012-96f15f27ff5d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:13.714820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a6355-cbb2-483a-8012-96f15f27ff5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:13.725070 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36dc2dd-568e-4b85-99eb-d22bdfd6b3a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:13.741789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36dc2dd-568e-4b85-99eb-d22bdfd6b3a1 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:13.792923 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 896/1793] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:13.793885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dc2dd-568e-4b85-99eb-d22bdfd6b3a1 service nova] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:13.804666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-5e6a6355-cbb2-483a-8012-96f15f27ff5d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:13.807301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5e6a6355-cbb2-483a-8012-96f15f27ff5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:13.835578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dc2dd-568e-4b85-99eb-d22bdfd6b3a1 service nova] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:14.069970 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b2303-778b-4e98-8ce8-202b74cea423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:14.071953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b2303-778b-4e98-8ce8-202b74cea423 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:14.073716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b2303-778b-4e98-8ce8-202b74cea423 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:14.076903 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b2303-778b-4e98-8ce8-202b74cea423 admin admin] PATH_INFO: `/v3/projects/cff6adeadf8f4a80becd517b88133d3c/users/9045f10eb4954c5791e5fce5f69b1d82/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:14.100120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 898/1794] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:14.103105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728899b1-83f4-46a3-b45e-515ad95a6f68 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:14.103940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728899b1-83f4-46a3-b45e-515ad95a6f68 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:14.104592 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728899b1-83f4-46a3-b45e-515ad95a6f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:14.145620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b2303-778b-4e98-8ce8-202b74cea423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cff6adeadf8f4a80becd517b88133d3c, user_id=9045f10eb4954c5791e5fce5f69b1d82, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:14.150675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b2303-778b-4e98-8ce8-202b74cea423 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:14.228352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 897/1795] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:13 2022] PUT /identity/v3/projects/cff6adeadf8f4a80becd517b88133d3c/users/9045f10eb4954c5791e5fce5f69b1d82/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:14.228352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54101190-3a1f-4815-b7b2-a9a5596e5d47 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:14.228352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54101190-3a1f-4815-b7b2-a9a5596e5d47 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:14.228352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54101190-3a1f-4815-b7b2-a9a5596e5d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:14.228779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-728899b1-83f4-46a3-b45e-515ad95a6f68 None None] MFA Rules not processed for user `87b956accf5849a7a8f73de53064948e`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:14.228779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-728899b1-83f4-46a3-b45e-515ad95a6f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:14.300088 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-54101190-3a1f-4815-b7b2-a9a5596e5d47 None None] MFA Rules not processed for user `a5adec9924444e29b8964f7969452592`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:14.310393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-54101190-3a1f-4815-b7b2-a9a5596e5d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:14.546691 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 899/1796] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 445 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:14.547695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728899b1-83f4-46a3-b45e-515ad95a6f68 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:14.577632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 898/1797] 10.222.0.57 () {58 vars in 1015 bytes} [Thu Jul 14 17:52:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:14.579044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b470c9-d2bf-4925-bd46-3ce7787ca82a None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:14.594809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b470c9-d2bf-4925-bd46-3ce7787ca82a None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:14.595115 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b470c9-d2bf-4925-bd46-3ce7787ca82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:14.727339 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728899b1-83f4-46a3-b45e-515ad95a6f68 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:14.736472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-17b470c9-d2bf-4925-bd46-3ce7787ca82a None None] MFA Rules not processed for user `9045f10eb4954c5791e5fce5f69b1d82`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:14.738471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-17b470c9-d2bf-4925-bd46-3ce7787ca82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:14.756083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faccc287-16e3-40b9-9964-b895928a9746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:14.766412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccc287-16e3-40b9-9964-b895928a9746 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:14.767314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccc287-16e3-40b9-9964-b895928a9746 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:14.774562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccc287-16e3-40b9-9964-b895928a9746 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:14.777239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faccc287-16e3-40b9-9964-b895928a9746 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:52:14.778668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faccc287-16e3-40b9-9964-b895928a9746 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:14.784314 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:14.784938 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faccc287-16e3-40b9-9964-b895928a9746 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:14.840953 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 900/1798] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:52:14.842183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faccc287-16e3-40b9-9964-b895928a9746 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:14.855862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faccc287-16e3-40b9-9964-b895928a9746 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:14.863699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43a807-e59c-4f9f-9a3f-5bd89492e23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:14.865381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43a807-e59c-4f9f-9a3f-5bd89492e23b admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:14.865821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43a807-e59c-4f9f-9a3f-5bd89492e23b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:14.866938 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be43a807-e59c-4f9f-9a3f-5bd89492e23b admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:14.869820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43a807-e59c-4f9f-9a3f-5bd89492e23b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:14.873794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be43a807-e59c-4f9f-9a3f-5bd89492e23b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:14.929853 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 901/1799] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:14 2022] POST /identity/v3/projects => generated 401 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:14.930935 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43a807-e59c-4f9f-9a3f-5bd89492e23b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:14.937201 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-be43a807-e59c-4f9f-9a3f-5bd89492e23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:14.957059 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:15.150367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be43a807-e59c-4f9f-9a3f-5bd89492e23b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:15.167865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 899/1800] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 589 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:15.168975 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bd9dc-3665-4ca6-b6fb-204311c87231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:15.169936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bd9dc-3665-4ca6-b6fb-204311c87231 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:15.171799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b470c9-d2bf-4925-bd46-3ce7787ca82a None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:15.174548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bd9dc-3665-4ca6-b6fb-204311c87231 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:15.176370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bd9dc-3665-4ca6-b6fb-204311c87231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:15.186256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3bd9dc-3665-4ca6-b6fb-204311c87231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:15.188269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3bd9dc-3665-4ca6-b6fb-204311c87231 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:15.201975 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 902/1801] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:15.204442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b470c9-d2bf-4925-bd46-3ce7787ca82a None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:15.204688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bd9dc-3665-4ca6-b6fb-204311c87231 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:15.360734 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bd9dc-3665-4ca6-b6fb-204311c87231 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:15.383125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9ad88a-8283-475a-a966-86dddc015b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:15.387371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ad88a-8283-475a-a966-86dddc015b8e admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:15.387702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ad88a-8283-475a-a966-86dddc015b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:15.388006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9ad88a-8283-475a-a966-86dddc015b8e admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:15.388757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ad88a-8283-475a-a966-86dddc015b8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:15.401299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9ad88a-8283-475a-a966-86dddc015b8e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:15.419915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20356ea2-c606-4706-b516-266947d3494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:15.420669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356ea2-c606-4706-b516-266947d3494c admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:15.421880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356ea2-c606-4706-b516-266947d3494c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:15.421880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20356ea2-c606-4706-b516-266947d3494c admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:15.422169 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20356ea2-c606-4706-b516-266947d3494c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:15.424869 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20356ea2-c606-4706-b516-266947d3494c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:15.476655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 903/1802] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:15 2022] POST /identity/v3/users => generated 307 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:15.476655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293773f-c147-4d53-b229-67941bd0b93c None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:15.476655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293773f-c147-4d53-b229-67941bd0b93c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:15.488332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293773f-c147-4d53-b229-67941bd0b93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:15.492776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 900/1803] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:15 2022] POST /identity/v3/projects => generated 365 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:15.497008 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20356ea2-c606-4706-b516-266947d3494c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:15.553130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-7293773f-c147-4d53-b229-67941bd0b93c None None] MFA Rules not processed for user `84c38f6ff5af4ba79c80fb581baacb0b`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:15.555011 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7293773f-c147-4d53-b229-67941bd0b93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:15.721225 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20356ea2-c606-4706-b516-266947d3494c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:15.730783 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c5d15-2493-45ba-80a9-22f2df1f456b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:15.731698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c5d15-2493-45ba-80a9-22f2df1f456b admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:15.732254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c5d15-2493-45ba-80a9-22f2df1f456b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:15.732642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c5d15-2493-45ba-80a9-22f2df1f456b admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:15.751369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 904/1804] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:15.752812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c5d15-2493-45ba-80a9-22f2df1f456b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:15.756012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c5d15-2493-45ba-80a9-22f2df1f456b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:15.756231 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93b7d4-d806-476a-9765-bca74451341e None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:15.756688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93b7d4-d806-476a-9765-bca74451341e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:15.757815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa93b7d4-d806-476a-9765-bca74451341e None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:15.764205 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 905/1805] 10.222.0.64 () {54 vars in 912 bytes} [Thu Jul 14 17:52:15 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:52:15.766171 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93b7d4-d806-476a-9765-bca74451341e None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:15.818909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa93b7d4-d806-476a-9765-bca74451341e None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:15.844263 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 901/1806] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:15 2022] POST /identity/v3/projects => generated 371 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:15.850049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c5d15-2493-45ba-80a9-22f2df1f456b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:15.863140 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa99de-056d-46a6-a652-4e8314198ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:15.863885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa99de-056d-46a6-a652-4e8314198ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:15.870596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa99de-056d-46a6-a652-4e8314198ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:15.871891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa99de-056d-46a6-a652-4e8314198ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:15.874627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa99de-056d-46a6-a652-4e8314198ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:15.879162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa99de-056d-46a6-a652-4e8314198ec0 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:15.906565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:15.928578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 906/1807] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:15 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:15.933392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d469364-5c27-42af-80ea-19bffff30d64 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:15.934858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d469364-5c27-42af-80ea-19bffff30d64 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:15.936126 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d469364-5c27-42af-80ea-19bffff30d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:15.989816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-3d469364-5c27-42af-80ea-19bffff30d64 None None] MFA Rules not processed for user `a5adec9924444e29b8964f7969452592`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:16.002594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3d469364-5c27-42af-80ea-19bffff30d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:16.078957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:16.123624 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c5d15-2493-45ba-80a9-22f2df1f456b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:16.133291 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d0832-4b3b-461b-a1c2-92f9b02db8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:16.135482 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d0832-4b3b-461b-a1c2-92f9b02db8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:16.135877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d0832-4b3b-461b-a1c2-92f9b02db8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:16.136356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d0832-4b3b-461b-a1c2-92f9b02db8b1 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:16.137252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d0832-4b3b-461b-a1c2-92f9b02db8b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:16.139244 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d0832-4b3b-461b-a1c2-92f9b02db8b1 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:16.177377 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 902/1808] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:15 2022] POST /identity/v3/users => generated 283 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:16.178698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d0832-4b3b-461b-a1c2-92f9b02db8b1 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:16.179562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 907/1809] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 17:52:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:16.180757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d469364-5c27-42af-80ea-19bffff30d64 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:16.210904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d469364-5c27-42af-80ea-19bffff30d64 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:16.225617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d0832-4b3b-461b-a1c2-92f9b02db8b1 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:16.234729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3b8f9-5198-46a9-a562-2b85e92a5b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:16.235774 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b8f9-5198-46a9-a562-2b85e92a5b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:16.236184 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b8f9-5198-46a9-a562-2b85e92a5b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:16.236585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3b8f9-5198-46a9-a562-2b85e92a5b21 admin admin] PATH_INFO: `/v3/projects/781fae064b23446198039f4990070ebe/users/343192048b994ee39d3dbb1026f034d5/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:16.238863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824dec3-c27b-4bb7-befd-0879ab260be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:16.240309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824dec3-c27b-4bb7-befd-0879ab260be4 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:16.240761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824dec3-c27b-4bb7-befd-0879ab260be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:16.241781 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824dec3-c27b-4bb7-befd-0879ab260be4 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:16.242728 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824dec3-c27b-4bb7-befd-0879ab260be4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:16.245759 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824dec3-c27b-4bb7-befd-0879ab260be4 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:16.287054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3b8f9-5198-46a9-a562-2b85e92a5b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=781fae064b23446198039f4990070ebe, user_id=343192048b994ee39d3dbb1026f034d5, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:16.292992 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d3b8f9-5198-46a9-a562-2b85e92a5b21 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:16.317654 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 903/1810] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:16 2022] POST /identity/v3/users => generated 292 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:16.319589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824dec3-c27b-4bb7-befd-0879ab260be4 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:16.327856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 908/1811] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:16 2022] PUT /identity/v3/projects/781fae064b23446198039f4990070ebe/users/343192048b994ee39d3dbb1026f034d5/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:16.328958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3b8f9-5198-46a9-a562-2b85e92a5b21 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:16.337485 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-68d3b8f9-5198-46a9-a562-2b85e92a5b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:16.370129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824dec3-c27b-4bb7-befd-0879ab260be4 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:16.511482 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d3b8f9-5198-46a9-a562-2b85e92a5b21 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:16.528284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020dd10a-4394-4230-a9fd-0f9662b4e535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:16.531020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020dd10a-4394-4230-a9fd-0f9662b4e535 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:16.534130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020dd10a-4394-4230-a9fd-0f9662b4e535 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:16.535853 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020dd10a-4394-4230-a9fd-0f9662b4e535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:16.566166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020dd10a-4394-4230-a9fd-0f9662b4e535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:16.568580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020dd10a-4394-4230-a9fd-0f9662b4e535 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:16.604345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 909/1812] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:16.605975 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc2a67-1036-48c4-8fb9-284927878335 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:16.606598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc2a67-1036-48c4-8fb9-284927878335 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:16.609133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc2a67-1036-48c4-8fb9-284927878335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:16.623228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554534b1-fa1d-474c-a50f-bf456b683f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:16.623788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554534b1-fa1d-474c-a50f-bf456b683f6c admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:16.626120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554534b1-fa1d-474c-a50f-bf456b683f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:16.626425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554534b1-fa1d-474c-a50f-bf456b683f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:16.627199 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554534b1-fa1d-474c-a50f-bf456b683f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:16.628871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554534b1-fa1d-474c-a50f-bf456b683f6c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:16.666527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-a6cc2a67-1036-48c4-8fb9-284927878335 None None] MFA Rules not processed for user `c4178d34dc114f46af725c9dc362d224`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:16.668866 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a6cc2a67-1036-48c4-8fb9-284927878335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:16.694592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 904/1813] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:16.695660 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554534b1-fa1d-474c-a50f-bf456b683f6c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:16.751369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554534b1-fa1d-474c-a50f-bf456b683f6c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:16.765956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949ed8d-3e7f-429b-a13b-a4770a2d1f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:16.766798 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949ed8d-3e7f-429b-a13b-a4770a2d1f8e admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:16.767087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949ed8d-3e7f-429b-a13b-a4770a2d1f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:16.769048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949ed8d-3e7f-429b-a13b-a4770a2d1f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:16.769935 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949ed8d-3e7f-429b-a13b-a4770a2d1f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:16.777603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949ed8d-3e7f-429b-a13b-a4770a2d1f8e admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:16.844698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 905/1814] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:16.846369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949ed8d-3e7f-429b-a13b-a4770a2d1f8e admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:16.900908 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 910/1815] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:16.901999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc2a67-1036-48c4-8fb9-284927878335 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:16.935374 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949ed8d-3e7f-429b-a13b-a4770a2d1f8e admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:16.940224 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc2a67-1036-48c4-8fb9-284927878335 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:16.960668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c8be9-7bd2-4519-88dd-0afa1f09f216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:16.970197 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c8be9-7bd2-4519-88dd-0afa1f09f216 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:16.970751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c8be9-7bd2-4519-88dd-0afa1f09f216 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:16.971775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078e3c9-0f75-4419-b884-4882411a3bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:16.972304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c8be9-7bd2-4519-88dd-0afa1f09f216 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:16.974381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c8be9-7bd2-4519-88dd-0afa1f09f216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:16.980250 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078e3c9-0f75-4419-b884-4882411a3bdd admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:16.980542 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078e3c9-0f75-4419-b884-4882411a3bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:16.982678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1078e3c9-0f75-4419-b884-4882411a3bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:16.983502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078e3c9-0f75-4419-b884-4882411a3bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:16.988008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c8be9-7bd2-4519-88dd-0afa1f09f216 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:17.000331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1078e3c9-0f75-4419-b884-4882411a3bdd admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:17.023428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 911/1816] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:17.026452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c8be9-7bd2-4519-88dd-0afa1f09f216 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:17.058885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c8be9-7bd2-4519-88dd-0afa1f09f216 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:17.070359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 906/1817] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:16 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:17.075371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078e3c9-0f75-4419-b884-4882411a3bdd admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:17.097959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee3dd1-ff9e-436a-b1bc-c50e57888645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:17.097959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee3dd1-ff9e-436a-b1bc-c50e57888645 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:17.097959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee3dd1-ff9e-436a-b1bc-c50e57888645 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:17.097959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee3dd1-ff9e-436a-b1bc-c50e57888645 admin admin] PATH_INFO: `/v3/projects/6fc5f78b843d4c5b82da0930258e1eb0/users/6bb4904174884cb498f816f8fdad6756/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:17.120734 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1078e3c9-0f75-4419-b884-4882411a3bdd admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:17.130983 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7ac95-ee7a-4a66-ba43-4a2bc2dc66a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:17.133522 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7ac95-ee7a-4a66-ba43-4a2bc2dc66a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:17.133898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7ac95-ee7a-4a66-ba43-4a2bc2dc66a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:17.134443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7ac95-ee7a-4a66-ba43-4a2bc2dc66a1 admin admin] PATH_INFO: `/v3/projects/14459658bf134dd4b77274635c3e3c9e/users/d3c57988965b46259d09e531093f6e63/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:17.171558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee3dd1-ff9e-436a-b1bc-c50e57888645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fc5f78b843d4c5b82da0930258e1eb0, user_id=6bb4904174884cb498f816f8fdad6756, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:17.174264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee3dd1-ff9e-436a-b1bc-c50e57888645 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:17.179093 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7ac95-ee7a-4a66-ba43-4a2bc2dc66a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14459658bf134dd4b77274635c3e3c9e, user_id=d3c57988965b46259d09e531093f6e63, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:17.188615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd7ac95-ee7a-4a66-ba43-4a2bc2dc66a1 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:17.234137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 907/1818] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:17 2022] PUT /identity/v3/projects/14459658bf134dd4b77274635c3e3c9e/users/d3c57988965b46259d09e531093f6e63/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:17.234137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7ac95-ee7a-4a66-ba43-4a2bc2dc66a1 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:17.253082 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 912/1819] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:17 2022] PUT /identity/v3/projects/6fc5f78b843d4c5b82da0930258e1eb0/users/6bb4904174884cb498f816f8fdad6756/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:17.254532 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4397e-eb28-44ee-bea4-6507be6d23bf None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:17.254826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4397e-eb28-44ee-bea4-6507be6d23bf None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:17.255108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b4397e-eb28-44ee-bea4-6507be6d23bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:17.324632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-21b4397e-eb28-44ee-bea4-6507be6d23bf None None] MFA Rules not processed for user `d3c57988965b46259d09e531093f6e63`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:17.325890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-21b4397e-eb28-44ee-bea4-6507be6d23bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:17.373387 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:17.446235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:17.502812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7ac95-ee7a-4a66-ba43-4a2bc2dc66a1 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:17.564020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 913/1820] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:17 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:17.564020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4397e-eb28-44ee-bea4-6507be6d23bf None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:17.843484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae27c0-2406-499b-9637-4f01550b33b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:17.844575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae27c0-2406-499b-9637-4f01550b33b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:17.845305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae27c0-2406-499b-9637-4f01550b33b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:17.845893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae27c0-2406-499b-9637-4f01550b33b3 admin admin] PATH_INFO: `/v3/projects/781fae064b23446198039f4990070ebe/users/343192048b994ee39d3dbb1026f034d5/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:17.850209 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae27c0-2406-499b-9637-4f01550b33b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=781fae064b23446198039f4990070ebe, user_id=343192048b994ee39d3dbb1026f034d5, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:17.854987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae27c0-2406-499b-9637-4f01550b33b3 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:17.892052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 908/1821] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:17 2022] PUT /identity/v3/projects/781fae064b23446198039f4990070ebe/users/343192048b994ee39d3dbb1026f034d5/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:17.892052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae27c0-2406-499b-9637-4f01550b33b3 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:17.955740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b4397e-eb28-44ee-bea4-6507be6d23bf None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:18.156807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97444d5d-c45a-4edc-b87e-d61ede812940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:18.157513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97444d5d-c45a-4edc-b87e-d61ede812940 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:18.158740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97444d5d-c45a-4edc-b87e-d61ede812940 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:18.159003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97444d5d-c45a-4edc-b87e-d61ede812940 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:18.162015 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97444d5d-c45a-4edc-b87e-d61ede812940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:18.163856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97444d5d-c45a-4edc-b87e-d61ede812940 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:18.180655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae27c0-2406-499b-9637-4f01550b33b3 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:18.201694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e339861-81bb-4b46-ba76-2c54733118cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:18.202759 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e339861-81bb-4b46-ba76-2c54733118cb admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:18.212758 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 914/1822] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:17 2022] GET /identity/v3/roles => generated 1420 bytes in 651 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:18.217783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ae632-d7ee-4fb4-8fcc-562ee80370d5 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:18.218066 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ae632-d7ee-4fb4-8fcc-562ee80370d5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:18.218352 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ae632-d7ee-4fb4-8fcc-562ee80370d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:18.221773 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e339861-81bb-4b46-ba76-2c54733118cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:18.221958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e339861-81bb-4b46-ba76-2c54733118cb admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:18.222705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e339861-81bb-4b46-ba76-2c54733118cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:18.226104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e339861-81bb-4b46-ba76-2c54733118cb admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:18.283964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 909/1823] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:17 2022] POST /identity/v3/projects => generated 365 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:18.284824 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e339861-81bb-4b46-ba76-2c54733118cb admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:18.324911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-923ae632-d7ee-4fb4-8fcc-562ee80370d5 None None] MFA Rules not processed for user `343192048b994ee39d3dbb1026f034d5`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:18.327352 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-923ae632-d7ee-4fb4-8fcc-562ee80370d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:18.450917 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e339861-81bb-4b46-ba76-2c54733118cb admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:18.476833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c3748-3ac0-4e27-b0c9-887f9ca08b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:18.479473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c3748-3ac0-4e27-b0c9-887f9ca08b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:18.479764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c3748-3ac0-4e27-b0c9-887f9ca08b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:18.480072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c3748-3ac0-4e27-b0c9-887f9ca08b66 admin admin] PATH_INFO: `/v3/projects/6fc5f78b843d4c5b82da0930258e1eb0/users/6bb4904174884cb498f816f8fdad6756/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:18.496746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c3748-3ac0-4e27-b0c9-887f9ca08b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fc5f78b843d4c5b82da0930258e1eb0, user_id=6bb4904174884cb498f816f8fdad6756, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:18.499468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c3748-3ac0-4e27-b0c9-887f9ca08b66 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:18.526700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:18.562128 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 910/1824] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:18 2022] PUT /identity/v3/projects/6fc5f78b843d4c5b82da0930258e1eb0/users/6bb4904174884cb498f816f8fdad6756/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:18.581502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c3748-3ac0-4e27-b0c9-887f9ca08b66 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:18.595546 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:18.600124 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 915/1825] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:18 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:18.602339 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e8bf9-e2b3-4054-aa34-220aef2d0682 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:18.602964 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e8bf9-e2b3-4054-aa34-220aef2d0682 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:18.602964 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e8bf9-e2b3-4054-aa34-220aef2d0682 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:18.645540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 916/1826] 10.222.0.64 () {54 vars in 921 bytes} [Thu Jul 14 17:52:18 2022] GET /identity => generated 270 bytes in 45 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:52:18.650943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1f5e-951b-4460-af7b-e306270218cd None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:18.651296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1f5e-951b-4460-af7b-e306270218cd None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:18.651634 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb1f5e-951b-4460-af7b-e306270218cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:18.807226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-4ffb1f5e-951b-4460-af7b-e306270218cd None None] MFA Rules not processed for user `6bb4904174884cb498f816f8fdad6756`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:18.817100 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4ffb1f5e-951b-4460-af7b-e306270218cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:18.883155 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c3748-3ac0-4e27-b0c9-887f9ca08b66 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:18.906841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082d25d-5492-4996-8b46-bc791435b22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:18.907548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082d25d-5492-4996-8b46-bc791435b22e admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:18.907835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082d25d-5492-4996-8b46-bc791435b22e admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:18.910221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082d25d-5492-4996-8b46-bc791435b22e admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:18.910980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082d25d-5492-4996-8b46-bc791435b22e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:18.918141 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082d25d-5492-4996-8b46-bc791435b22e admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:18.964496 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 911/1827] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:18 2022] POST /identity/v3/users => generated 289 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:18.965730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082d25d-5492-4996-8b46-bc791435b22e admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:18.969301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d082d25d-5492-4996-8b46-bc791435b22e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:19.055103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 917/1828] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:18 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:19.060277 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5559ab6-e10e-4bc4-a59a-5c3245bce9ae None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:19.060679 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5559ab6-e10e-4bc4-a59a-5c3245bce9ae None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:19.060970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5559ab6-e10e-4bc4-a59a-5c3245bce9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:19.131291 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b5559ab6-e10e-4bc4-a59a-5c3245bce9ae None None] MFA Rules not processed for user `6423f54152064b909d03c0908d636993`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:19.133307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b5559ab6-e10e-4bc4-a59a-5c3245bce9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:19.184645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082d25d-5492-4996-8b46-bc791435b22e admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:19.190788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd1791-e1f4-49ed-a18e-dd1d2db7bc60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:19.191641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd1791-e1f4-49ed-a18e-dd1d2db7bc60 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:19.192123 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd1791-e1f4-49ed-a18e-dd1d2db7bc60 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:19.192488 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fd1791-e1f4-49ed-a18e-dd1d2db7bc60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:19.348759 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd1791-e1f4-49ed-a18e-dd1d2db7bc60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:19.350233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fd1791-e1f4-49ed-a18e-dd1d2db7bc60 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:19.389584 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 918/1829] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Jul 14 17:52:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:19.395878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5559ab6-e10e-4bc4-a59a-5c3245bce9ae None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:19.402074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 912/1830] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:18 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:19.403120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd1791-e1f4-49ed-a18e-dd1d2db7bc60 service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:19.413988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-78fd1791-e1f4-49ed-a18e-dd1d2db7bc60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:19.441234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5559ab6-e10e-4bc4-a59a-5c3245bce9ae None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:19.458579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0781c-a26b-414e-ac02-18e917291775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:19.463858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0781c-a26b-414e-ac02-18e917291775 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:19.465176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0781c-a26b-414e-ac02-18e917291775 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:19.467238 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b0781c-a26b-414e-ac02-18e917291775 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:19.472186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fd1791-e1f4-49ed-a18e-dd1d2db7bc60 service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:19.478168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0781c-a26b-414e-ac02-18e917291775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:19.486957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b0781c-a26b-414e-ac02-18e917291775 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:19.500108 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953c091c-3b4c-488b-843f-93fff534636d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:19.505254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c091c-3b4c-488b-843f-93fff534636d service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:19.506551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c091c-3b4c-488b-843f-93fff534636d service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:19.509215 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953c091c-3b4c-488b-843f-93fff534636d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:19.537326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 919/1831] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:19.546159 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0781c-a26b-414e-ac02-18e917291775 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:19.564517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c091c-3b4c-488b-843f-93fff534636d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:19.569167 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953c091c-3b4c-488b-843f-93fff534636d service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:19.590349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b0781c-a26b-414e-ac02-18e917291775 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:19.602377 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 913/1832] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:19 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:19.614559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a25f2-2783-41c4-9262-bf7286f9506b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:19.615639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a25f2-2783-41c4-9262-bf7286f9506b admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:19.616030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a25f2-2783-41c4-9262-bf7286f9506b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:19.616411 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a25f2-2783-41c4-9262-bf7286f9506b admin admin] PATH_INFO: `/v3/projects/4de5a78d0a4349df9ce5669f9c24a24f/users/e558e231649e4eefb0d198fb54183d80/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:19.661132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a25f2-2783-41c4-9262-bf7286f9506b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de5a78d0a4349df9ce5669f9c24a24f, user_id=e558e231649e4eefb0d198fb54183d80, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:19.670204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a25f2-2783-41c4-9262-bf7286f9506b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:19.726962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28a22f-ab1c-47eb-9af9-7c06d68f0b76 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:19.727261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28a22f-ab1c-47eb-9af9-7c06d68f0b76 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:19.727543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28a22f-ab1c-47eb-9af9-7c06d68f0b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:19.766301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 920/1833] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:19 2022] PUT /identity/v3/projects/4de5a78d0a4349df9ce5669f9c24a24f/users/e558e231649e4eefb0d198fb54183d80/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:19.770362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a25f2-2783-41c4-9262-bf7286f9506b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:19.823240 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-ac28a22f-ab1c-47eb-9af9-7c06d68f0b76 None None] MFA Rules not processed for user `52c794c11ab74f9ca9bad75857043a62`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:19.832646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ac28a22f-ab1c-47eb-9af9-7c06d68f0b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:19.957362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:19.999643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:20.075379 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a25f2-2783-41c4-9262-bf7286f9506b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:20.087809 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d26aa-2466-47b7-97a3-1e7eff82f572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:20.089131 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d26aa-2466-47b7-97a3-1e7eff82f572 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:20.091645 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d26aa-2466-47b7-97a3-1e7eff82f572 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:20.102108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d26aa-2466-47b7-97a3-1e7eff82f572 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:20.103981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d26aa-2466-47b7-97a3-1e7eff82f572 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:20.105735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d26aa-2466-47b7-97a3-1e7eff82f572 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:20.153005 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 921/1834] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:20.154596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d26aa-2466-47b7-97a3-1e7eff82f572 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:20.160039 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 914/1835] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 431 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:52:20.161660 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28a22f-ab1c-47eb-9af9-7c06d68f0b76 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:20.230790 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28a22f-ab1c-47eb-9af9-7c06d68f0b76 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:20.246212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f13ab-7d1d-4344-bf1c-628e01a2f5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:20.250766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f13ab-7d1d-4344-bf1c-628e01a2f5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:20.251184 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f13ab-7d1d-4344-bf1c-628e01a2f5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:20.253073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517f13ab-7d1d-4344-bf1c-628e01a2f5fd admin admin] PATH_INFO: `/v3/projects/4de5a78d0a4349df9ce5669f9c24a24f/users/e558e231649e4eefb0d198fb54183d80/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:20.264897 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f13ab-7d1d-4344-bf1c-628e01a2f5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4de5a78d0a4349df9ce5669f9c24a24f, user_id=e558e231649e4eefb0d198fb54183d80, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:20.268784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517f13ab-7d1d-4344-bf1c-628e01a2f5fd admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:20.325428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 915/1836] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:20 2022] PUT /identity/v3/projects/4de5a78d0a4349df9ce5669f9c24a24f/users/e558e231649e4eefb0d198fb54183d80/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:20.326492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f13ab-7d1d-4344-bf1c-628e01a2f5fd admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:20.334062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d26aa-2466-47b7-97a3-1e7eff82f572 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:20.633214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c56674-1429-46e3-8ab6-1b4681a8f927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:20.633924 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c56674-1429-46e3-8ab6-1b4681a8f927 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:20.634212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c56674-1429-46e3-8ab6-1b4681a8f927 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:20.634490 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c56674-1429-46e3-8ab6-1b4681a8f927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:20.666996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517f13ab-7d1d-4344-bf1c-628e01a2f5fd admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:20.688107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8a77d-38ae-447f-9aa3-dab55d962220 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:20.698307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8a77d-38ae-447f-9aa3-dab55d962220 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:20.699771 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8a77d-38ae-447f-9aa3-dab55d962220 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:20.700183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8a77d-38ae-447f-9aa3-dab55d962220 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:20.737128 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8a77d-38ae-447f-9aa3-dab55d962220 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:20.739520 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c8a77d-38ae-447f-9aa3-dab55d962220 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:20.765676 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 916/1837] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:20.767311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf98f12-077f-4db1-b0f5-4ec8446aa3cd None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:20.767791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf98f12-077f-4db1-b0f5-4ec8446aa3cd None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:20.768304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf98f12-077f-4db1-b0f5-4ec8446aa3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:20.850478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4bf98f12-077f-4db1-b0f5-4ec8446aa3cd None None] MFA Rules not processed for user `e558e231649e4eefb0d198fb54183d80`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:20.853630 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4bf98f12-077f-4db1-b0f5-4ec8446aa3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:20.858461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c56674-1429-46e3-8ab6-1b4681a8f927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:20.860032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c56674-1429-46e3-8ab6-1b4681a8f927 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:20.898713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 922/1838] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:20 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 745 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:20.899659 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c56674-1429-46e3-8ab6-1b4681a8f927 service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:20.923936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c56674-1429-46e3-8ab6-1b4681a8f927 service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:20.936197 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4efd4e-e5ea-4021-b866-6cf75252ae3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:20.937321 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4efd4e-e5ea-4021-b866-6cf75252ae3b service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:20.937509 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4efd4e-e5ea-4021-b866-6cf75252ae3b service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:20.954915 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4efd4e-e5ea-4021-b866-6cf75252ae3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:21.139036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:21.194435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 917/1839] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:20 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:21.259742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4efd4e-e5ea-4021-b866-6cf75252ae3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:21.260829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4efd4e-e5ea-4021-b866-6cf75252ae3b service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:21.297245 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 923/1840] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:21.593046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf98f12-077f-4db1-b0f5-4ec8446aa3cd None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:21.624778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf98f12-077f-4db1-b0f5-4ec8446aa3cd None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:21.632357 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6ff01-f739-4d6f-b6e0-592d63deaa88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:21.633422 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6ff01-f739-4d6f-b6e0-592d63deaa88 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:21.633983 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6ff01-f739-4d6f-b6e0-592d63deaa88 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:21.634669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6ff01-f739-4d6f-b6e0-592d63deaa88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:21.677320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6ff01-f739-4d6f-b6e0-592d63deaa88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:21.681700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6ff01-f739-4d6f-b6e0-592d63deaa88 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:21.729161 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 918/1841] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:22.328080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4efd4e-e5ea-4021-b866-6cf75252ae3b service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:22.367733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4efd4e-e5ea-4021-b866-6cf75252ae3b service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:22.389072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c143a-d72f-43a7-8f73-670adb2a25a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:22.390700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c143a-d72f-43a7-8f73-670adb2a25a0 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:22.394689 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c143a-d72f-43a7-8f73-670adb2a25a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:22.395383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c143a-d72f-43a7-8f73-670adb2a25a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:22.631432 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c143a-d72f-43a7-8f73-670adb2a25a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:22.633548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c143a-d72f-43a7-8f73-670adb2a25a0 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:22.662040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 924/1842] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:22 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:26.343964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6ff01-f739-4d6f-b6e0-592d63deaa88 service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:26.364320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:26.412484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6ff01-f739-4d6f-b6e0-592d63deaa88 service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:26.417767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5411df-9934-45b1-9830-2144687ce20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:26.418665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5411df-9934-45b1-9830-2144687ce20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:26.418949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5411df-9934-45b1-9830-2144687ce20b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:26.419227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5411df-9934-45b1-9830-2144687ce20b admin admin] PATH_INFO: `/v3/users/d3c57988965b46259d09e531093f6e63` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:26.420651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5411df-9934-45b1-9830-2144687ce20b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3c57988965b46259d09e531093f6e63)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:26.424143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5411df-9934-45b1-9830-2144687ce20b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:26.480931 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1d5411df-9934-45b1-9830-2144687ce20b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3c57988965b46259d09e531093f6e63'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:26.505931 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1d5411df-9934-45b1-9830-2144687ce20b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3c57988965b46259d09e531093f6e63'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:26.506948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1d5411df-9934-45b1-9830-2144687ce20b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3c57988965b46259d09e531093f6e63'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:26.518563 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1d5411df-9934-45b1-9830-2144687ce20b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3c57988965b46259d09e531093f6e63'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:26.519280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1d5411df-9934-45b1-9830-2144687ce20b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3c57988965b46259d09e531093f6e63'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:26.547989 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 919/1843] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:26 2022] DELETE /identity/v3/users/d3c57988965b46259d09e531093f6e63 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:26.551397 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a85a55-14f8-4636-8fc2-d91dfb8b818b None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:26.552326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a85a55-14f8-4636-8fc2-d91dfb8b818b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:26.552761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a85a55-14f8-4636-8fc2-d91dfb8b818b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:26.583388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-01a85a55-14f8-4636-8fc2-d91dfb8b818b None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:26.584964 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-01a85a55-14f8-4636-8fc2-d91dfb8b818b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:26.677262 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 925/1844] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:52:26.687241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5411df-9934-45b1-9830-2144687ce20b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:26.688948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1d5411df-9934-45b1-9830-2144687ce20b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:26.760763 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5411df-9934-45b1-9830-2144687ce20b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:26.764674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678adc6a-b763-4624-885c-3ab766f473c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:26.765176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678adc6a-b763-4624-885c-3ab766f473c8 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:26.765738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678adc6a-b763-4624-885c-3ab766f473c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:26.766031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678adc6a-b763-4624-885c-3ab766f473c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:26.777105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678adc6a-b763-4624-885c-3ab766f473c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:26.778550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678adc6a-b763-4624-885c-3ab766f473c8 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:26.790183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 920/1845] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:27.082081 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a85a55-14f8-4636-8fc2-d91dfb8b818b None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:27.083534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-01a85a55-14f8-4636-8fc2-d91dfb8b818b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:27.096880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a85a55-14f8-4636-8fc2-d91dfb8b818b None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:27.102097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:27.102970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:27.103253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:27.103522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] PATH_INFO: `/v3/projects/14459658bf134dd4b77274635c3e3c9e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:27.104773 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14459658bf134dd4b77274635c3e3c9e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:27.106708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:27.154796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14459658bf134dd4b77274635c3e3c9e', 'request_id': 'req-d810ecd8-851d-4ea0-9cde-166ed9dfc125'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.162158 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14459658bf134dd4b77274635c3e3c9e', 'request_id': 'req-d810ecd8-851d-4ea0-9cde-166ed9dfc125'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.168522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14459658bf134dd4b77274635c3e3c9e', 'request_id': 'req-d810ecd8-851d-4ea0-9cde-166ed9dfc125'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.172409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:27.181469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] The token cache is being invalidate because project 14459658bf134dd4b77274635c3e3c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:27.181979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.182981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.184694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 926/1846] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:27 2022] DELETE /identity/v3/projects/14459658bf134dd4b77274635c3e3c9e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:27.189090 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678adc6a-b763-4624-885c-3ab766f473c8 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:27.199634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678adc6a-b763-4624-885c-3ab766f473c8 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:27.237745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:27.262120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637dcfa-c14a-4fe8-acd7-d0d370082bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:27.262977 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637dcfa-c14a-4fe8-acd7-d0d370082bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:27.263406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637dcfa-c14a-4fe8-acd7-d0d370082bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:27.263685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637dcfa-c14a-4fe8-acd7-d0d370082bf7 admin admin] PATH_INFO: `/v3/users/e558e231649e4eefb0d198fb54183d80` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:27.264983 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637dcfa-c14a-4fe8-acd7-d0d370082bf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e558e231649e4eefb0d198fb54183d80)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:27.266920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637dcfa-c14a-4fe8-acd7-d0d370082bf7 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:27.319104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e637dcfa-c14a-4fe8-acd7-d0d370082bf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e558e231649e4eefb0d198fb54183d80'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.333806 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e637dcfa-c14a-4fe8-acd7-d0d370082bf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e558e231649e4eefb0d198fb54183d80'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.334606 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e637dcfa-c14a-4fe8-acd7-d0d370082bf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e558e231649e4eefb0d198fb54183d80'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.349261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e637dcfa-c14a-4fe8-acd7-d0d370082bf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e558e231649e4eefb0d198fb54183d80'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.350306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e637dcfa-c14a-4fe8-acd7-d0d370082bf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e558e231649e4eefb0d198fb54183d80'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.364516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 921/1847] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:27 2022] DELETE /identity/v3/users/e558e231649e4eefb0d198fb54183d80 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:27.696046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:27.697539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:27.712643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810ecd8-851d-4ea0-9cde-166ed9dfc125 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:27.792923 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744e3201-71fa-495d-970f-1196cb422eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:27.793813 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e3201-71fa-495d-970f-1196cb422eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:27.794097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e3201-71fa-495d-970f-1196cb422eac admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:27.794377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744e3201-71fa-495d-970f-1196cb422eac admin admin] PATH_INFO: `/v3/projects/4de5a78d0a4349df9ce5669f9c24a24f` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:27.795601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e3201-71fa-495d-970f-1196cb422eac admin admin] RBAC: Authorizing `identity:delete_project(project_id=4de5a78d0a4349df9ce5669f9c24a24f)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:27.797544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744e3201-71fa-495d-970f-1196cb422eac admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:27.849180 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-744e3201-71fa-495d-970f-1196cb422eac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4de5a78d0a4349df9ce5669f9c24a24f', 'request_id': 'req-744e3201-71fa-495d-970f-1196cb422eac'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.856266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-744e3201-71fa-495d-970f-1196cb422eac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4de5a78d0a4349df9ce5669f9c24a24f', 'request_id': 'req-744e3201-71fa-495d-970f-1196cb422eac'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.862518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-744e3201-71fa-495d-970f-1196cb422eac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4de5a78d0a4349df9ce5669f9c24a24f', 'request_id': 'req-744e3201-71fa-495d-970f-1196cb422eac'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.873660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-744e3201-71fa-495d-970f-1196cb422eac admin admin] The token cache is being invalidate because project 4de5a78d0a4349df9ce5669f9c24a24f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:27.873953 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-744e3201-71fa-495d-970f-1196cb422eac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.874636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-744e3201-71fa-495d-970f-1196cb422eac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:27.876273 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 927/1848] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:27 2022] DELETE /identity/v3/projects/4de5a78d0a4349df9ce5669f9c24a24f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:27.881375 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298dd47-81ef-41f4-b226-a0bfd20212bb None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:27.881674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298dd47-81ef-41f4-b226-a0bfd20212bb None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:27.881956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298dd47-81ef-41f4-b226-a0bfd20212bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:27.899703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-6298dd47-81ef-41f4-b226-a0bfd20212bb None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:27.900942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6298dd47-81ef-41f4-b226-a0bfd20212bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:27.962667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 922/1849] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:52:27.971320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e8545-5a15-43db-9636-5caaa4c37492 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:27.971619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e8545-5a15-43db-9636-5caaa4c37492 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:27.971905 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e8545-5a15-43db-9636-5caaa4c37492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:27.990480 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-9a8e8545-5a15-43db-9636-5caaa4c37492 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:27.991752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9a8e8545-5a15-43db-9636-5caaa4c37492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:28.001021 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 928/1850] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:28.005342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298dd47-81ef-41f4-b226-a0bfd20212bb None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:28.015441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6298dd47-81ef-41f4-b226-a0bfd20212bb None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:28.020134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb8fd2-32b3-479c-a584-734344573a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:28.020747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb8fd2-32b3-479c-a584-734344573a72 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:28.021038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb8fd2-32b3-479c-a584-734344573a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:28.021321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffb8fd2-32b3-479c-a584-734344573a72 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:28.022056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb8fd2-32b3-479c-a584-734344573a72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:52:28.022365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb8fd2-32b3-479c-a584-734344573a72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:28.024494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffb8fd2-32b3-479c-a584-734344573a72 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:28.036851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 923/1851] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:52:28.042030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e8545-5a15-43db-9636-5caaa4c37492 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:28.051739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e8545-5a15-43db-9636-5caaa4c37492 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:28.056643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fb017-6b2e-474f-8648-58476ff5a197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:28.057301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fb017-6b2e-474f-8648-58476ff5a197 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:28.057588 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fb017-6b2e-474f-8648-58476ff5a197 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:28.057870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8fb017-6b2e-474f-8648-58476ff5a197 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:28.058628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8fb017-6b2e-474f-8648-58476ff5a197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:28.060520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8fb017-6b2e-474f-8648-58476ff5a197 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:28.085829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 929/1852] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:28 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:28.090603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb8fd2-32b3-479c-a584-734344573a72 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:28.100553 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffb8fd2-32b3-479c-a584-734344573a72 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:28.145127 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:28.160832 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be7806-2cb6-4a0f-b566-051de79fc845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:28.161496 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7806-2cb6-4a0f-b566-051de79fc845 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:28.161779 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7806-2cb6-4a0f-b566-051de79fc845 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:28.162051 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be7806-2cb6-4a0f-b566-051de79fc845 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:28.162744 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be7806-2cb6-4a0f-b566-051de79fc845 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:28.165267 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59be7806-2cb6-4a0f-b566-051de79fc845 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:28.185212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 924/1853] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:28 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:52:28.189543 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fb017-6b2e-474f-8648-58476ff5a197 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:28.199703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8fb017-6b2e-474f-8648-58476ff5a197 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:28.204307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624e872-e00b-4022-b338-e6f3c18acfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:28.204965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624e872-e00b-4022-b338-e6f3c18acfc8 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:28.205246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624e872-e00b-4022-b338-e6f3c18acfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:28.205519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624e872-e00b-4022-b338-e6f3c18acfc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:28.206243 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624e872-e00b-4022-b338-e6f3c18acfc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:28.207820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c624e872-e00b-4022-b338-e6f3c18acfc8 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:28.218624 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 930/1854] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:28.222510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be7806-2cb6-4a0f-b566-051de79fc845 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:28.233086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be7806-2cb6-4a0f-b566-051de79fc845 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:28.237637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32618c8-ee4c-4add-868f-12558cb73006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:28.238291 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32618c8-ee4c-4add-868f-12558cb73006 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:28.238579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32618c8-ee4c-4add-868f-12558cb73006 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:28.238861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32618c8-ee4c-4add-868f-12558cb73006 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:28.239587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32618c8-ee4c-4add-868f-12558cb73006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:28.241212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32618c8-ee4c-4add-868f-12558cb73006 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:28.251021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 925/1855] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:28.255161 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624e872-e00b-4022-b338-e6f3c18acfc8 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:28.265233 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624e872-e00b-4022-b338-e6f3c18acfc8 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:28.269715 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a803e7-0359-4264-adf0-d2e80ce50cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:28.270643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a803e7-0359-4264-adf0-d2e80ce50cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:28.270876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a803e7-0359-4264-adf0-d2e80ce50cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:28.271155 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a803e7-0359-4264-adf0-d2e80ce50cb4 admin admin] PATH_INFO: `/v3/projects/2c7a2c14a1cc465896a924f2ebab809e/users/bbafbab4281e4a6eb168aa8e3ea151c2/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:28.286241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a803e7-0359-4264-adf0-d2e80ce50cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c7a2c14a1cc465896a924f2ebab809e, user_id=bbafbab4281e4a6eb168aa8e3ea151c2, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:28.289794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a803e7-0359-4264-adf0-d2e80ce50cb4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:28.302507 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 931/1856] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:28 2022] PUT /identity/v3/projects/2c7a2c14a1cc465896a924f2ebab809e/users/bbafbab4281e4a6eb168aa8e3ea151c2/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:28.312958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f584145-52dd-449f-87c9-733c4fa8e962 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:28.313245 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f584145-52dd-449f-87c9-733c4fa8e962 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:28.313524 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f584145-52dd-449f-87c9-733c4fa8e962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:28.339825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5f584145-52dd-449f-87c9-733c4fa8e962 None None] MFA Rules not processed for user `bbafbab4281e4a6eb168aa8e3ea151c2`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:28.341047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5f584145-52dd-449f-87c9-733c4fa8e962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:28.405622 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 926/1857] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:28.410284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a803e7-0359-4264-adf0-d2e80ce50cb4 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:28.431665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:28.477444 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a803e7-0359-4264-adf0-d2e80ce50cb4 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:28.490312 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f507843-c56c-4271-8443-7897955c263c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:28.491169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f507843-c56c-4271-8443-7897955c263c admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:28.491478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f507843-c56c-4271-8443-7897955c263c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:28.492259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f507843-c56c-4271-8443-7897955c263c admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:28.492514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f507843-c56c-4271-8443-7897955c263c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:28.494696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f507843-c56c-4271-8443-7897955c263c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:28.528208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 932/1858] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:28 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:28.530064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f584145-52dd-449f-87c9-733c4fa8e962 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:28.543907 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f584145-52dd-449f-87c9-733c4fa8e962 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:28.612963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1f6ae-b8ad-461d-8cf5-dc8d82a7db23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:28.613710 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1f6ae-b8ad-461d-8cf5-dc8d82a7db23 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:28.613914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1f6ae-b8ad-461d-8cf5-dc8d82a7db23 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:28.614489 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d1f6ae-b8ad-461d-8cf5-dc8d82a7db23 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:28.614903 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1f6ae-b8ad-461d-8cf5-dc8d82a7db23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:28.616834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d1f6ae-b8ad-461d-8cf5-dc8d82a7db23 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:28.639831 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 927/1859] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:28 2022] POST /identity/v3/users => generated 295 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:28.644279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f507843-c56c-4271-8443-7897955c263c admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:28.655992 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f507843-c56c-4271-8443-7897955c263c admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:28.660499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72852c4-13a9-4a54-b884-9c9701ee68d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:28.661169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72852c4-13a9-4a54-b884-9c9701ee68d3 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:28.661458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72852c4-13a9-4a54-b884-9c9701ee68d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:28.661741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72852c4-13a9-4a54-b884-9c9701ee68d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:28.662466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72852c4-13a9-4a54-b884-9c9701ee68d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:28.664118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72852c4-13a9-4a54-b884-9c9701ee68d3 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:28.674605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 933/1860] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:28.678646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1f6ae-b8ad-461d-8cf5-dc8d82a7db23 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:28.688921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d1f6ae-b8ad-461d-8cf5-dc8d82a7db23 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:28.693423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece4edc-08d0-48a3-b943-8f24fcb3f4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:28.694304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece4edc-08d0-48a3-b943-8f24fcb3f4e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:28.694589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece4edc-08d0-48a3-b943-8f24fcb3f4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:28.694863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece4edc-08d0-48a3-b943-8f24fcb3f4e5 admin admin] PATH_INFO: `/v3/projects/1d71574c4e51413e989675de41cf398d/users/2229a34f5b5648808850c6bcc09a8860/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:28.713709 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece4edc-08d0-48a3-b943-8f24fcb3f4e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d71574c4e51413e989675de41cf398d, user_id=2229a34f5b5648808850c6bcc09a8860, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:28.716430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece4edc-08d0-48a3-b943-8f24fcb3f4e5 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:28.729291 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 928/1861] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:28 2022] PUT /identity/v3/projects/1d71574c4e51413e989675de41cf398d/users/2229a34f5b5648808850c6bcc09a8860/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:28.733299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72852c4-13a9-4a54-b884-9c9701ee68d3 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:28.743243 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72852c4-13a9-4a54-b884-9c9701ee68d3 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:28.801569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c908c30-e38b-454f-9b0b-6c4daa34cc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:28.802252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c908c30-e38b-454f-9b0b-6c4daa34cc67 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:28.802544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c908c30-e38b-454f-9b0b-6c4daa34cc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:28.802825 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c908c30-e38b-454f-9b0b-6c4daa34cc67 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:28.803589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c908c30-e38b-454f-9b0b-6c4daa34cc67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:28.805741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c908c30-e38b-454f-9b0b-6c4daa34cc67 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:28.815955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 934/1862] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:28.821055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece4edc-08d0-48a3-b943-8f24fcb3f4e5 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:28.832058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece4edc-08d0-48a3-b943-8f24fcb3f4e5 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:28.844509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb94a23-3759-46dd-9a7b-638af3896ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:28.845590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb94a23-3759-46dd-9a7b-638af3896ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:28.846061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb94a23-3759-46dd-9a7b-638af3896ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:28.846419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb94a23-3759-46dd-9a7b-638af3896ad4 admin admin] PATH_INFO: `/v3/projects/1d71574c4e51413e989675de41cf398d/users/2229a34f5b5648808850c6bcc09a8860/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:28.848790 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb94a23-3759-46dd-9a7b-638af3896ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d71574c4e51413e989675de41cf398d, user_id=2229a34f5b5648808850c6bcc09a8860, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:28.851798 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb94a23-3759-46dd-9a7b-638af3896ad4 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:28.863429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 929/1863] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:28 2022] PUT /identity/v3/projects/1d71574c4e51413e989675de41cf398d/users/2229a34f5b5648808850c6bcc09a8860/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:28.875102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d291ae-a705-4f45-9426-ab5ac7789991 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:28.875496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d291ae-a705-4f45-9426-ab5ac7789991 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:28.875844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d291ae-a705-4f45-9426-ab5ac7789991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:28.904950 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-25d291ae-a705-4f45-9426-ab5ac7789991 None None] MFA Rules not processed for user `2229a34f5b5648808850c6bcc09a8860`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:28.906260 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-25d291ae-a705-4f45-9426-ab5ac7789991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:28.982618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 935/1864] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:28.994185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb94a23-3759-46dd-9a7b-638af3896ad4 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:28.995884 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-8bb94a23-3759-46dd-9a7b-638af3896ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:29.005901 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:29.058998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb94a23-3759-46dd-9a7b-638af3896ad4 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:29.062558 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7889999f-868a-4037-a21e-a95ad5d3f7d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:29.063602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889999f-868a-4037-a21e-a95ad5d3f7d7 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:29.063879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889999f-868a-4037-a21e-a95ad5d3f7d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:29.064172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7889999f-868a-4037-a21e-a95ad5d3f7d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:29.131475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889999f-868a-4037-a21e-a95ad5d3f7d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:29.133581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7889999f-868a-4037-a21e-a95ad5d3f7d7 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:29.143761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 930/1865] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:35.276151 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d291ae-a705-4f45-9426-ab5ac7789991 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:35.277487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-25d291ae-a705-4f45-9426-ab5ac7789991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:35.317354 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:35.370873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d291ae-a705-4f45-9426-ab5ac7789991 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:35.377093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea41c58-c37a-4a58-96c5-374087c39227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:35.378021 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea41c58-c37a-4a58-96c5-374087c39227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:35.378308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea41c58-c37a-4a58-96c5-374087c39227 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:35.378583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea41c58-c37a-4a58-96c5-374087c39227 admin admin] PATH_INFO: `/v3/users/87b956accf5849a7a8f73de53064948e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:35.379791 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea41c58-c37a-4a58-96c5-374087c39227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87b956accf5849a7a8f73de53064948e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:35.381933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea41c58-c37a-4a58-96c5-374087c39227 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:35.436742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4ea41c58-c37a-4a58-96c5-374087c39227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87b956accf5849a7a8f73de53064948e'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:35.449057 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4ea41c58-c37a-4a58-96c5-374087c39227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87b956accf5849a7a8f73de53064948e'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:35.456922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4ea41c58-c37a-4a58-96c5-374087c39227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87b956accf5849a7a8f73de53064948e'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:35.457576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4ea41c58-c37a-4a58-96c5-374087c39227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87b956accf5849a7a8f73de53064948e'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:35.464733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4ea41c58-c37a-4a58-96c5-374087c39227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87b956accf5849a7a8f73de53064948e'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:35.471136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 936/1866] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:35 2022] DELETE /identity/v3/users/87b956accf5849a7a8f73de53064948e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:35.475849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c39997-b087-4316-bb2c-1e34e70b36df None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:35.476401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c39997-b087-4316-bb2c-1e34e70b36df None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:35.476561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c39997-b087-4316-bb2c-1e34e70b36df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:35.494993 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-59c39997-b087-4316-bb2c-1e34e70b36df None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:35.496216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-59c39997-b087-4316-bb2c-1e34e70b36df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:35.562197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 931/1867] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:35.571350 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea41c58-c37a-4a58-96c5-374087c39227 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:35.572943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4ea41c58-c37a-4a58-96c5-374087c39227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:35.632478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea41c58-c37a-4a58-96c5-374087c39227 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:35.636227 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862cd50-6c43-49c4-b3f9-41f1b0f5d343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:35.636904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862cd50-6c43-49c4-b3f9-41f1b0f5d343 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:35.637191 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862cd50-6c43-49c4-b3f9-41f1b0f5d343 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:35.637471 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3862cd50-6c43-49c4-b3f9-41f1b0f5d343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:35.646631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862cd50-6c43-49c4-b3f9-41f1b0f5d343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:35.648095 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3862cd50-6c43-49c4-b3f9-41f1b0f5d343 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:35.658002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 937/1868] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:36.020289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c39997-b087-4316-bb2c-1e34e70b36df None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:36.021757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-59c39997-b087-4316-bb2c-1e34e70b36df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:36.034305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c39997-b087-4316-bb2c-1e34e70b36df None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:36.038990 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:36.039871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:36.040185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:36.040471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] PATH_INFO: `/v3/projects/df34a1d1b06f46fba7c5a7963e5019a8` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:36.041674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df34a1d1b06f46fba7c5a7963e5019a8)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:36.044403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:36.092754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df34a1d1b06f46fba7c5a7963e5019a8', 'request_id': 'req-ba7a1a05-85e2-4a92-9139-66509c2ff328'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.102386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df34a1d1b06f46fba7c5a7963e5019a8', 'request_id': 'req-ba7a1a05-85e2-4a92-9139-66509c2ff328'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.104087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:36.110159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df34a1d1b06f46fba7c5a7963e5019a8', 'request_id': 'req-ba7a1a05-85e2-4a92-9139-66509c2ff328'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.122784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] The token cache is being invalidate because project df34a1d1b06f46fba7c5a7963e5019a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:36.123069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.123739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.125308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 932/1869] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:36 2022] DELETE /identity/v3/projects/df34a1d1b06f46fba7c5a7963e5019a8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:36.129711 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862cd50-6c43-49c4-b3f9-41f1b0f5d343 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:36.140037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3862cd50-6c43-49c4-b3f9-41f1b0f5d343 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:36.173744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:36.202612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6431fd-0cc3-4719-b5a1-16a3a0c5ecd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:36.203516 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6431fd-0cc3-4719-b5a1-16a3a0c5ecd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:36.203811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6431fd-0cc3-4719-b5a1-16a3a0c5ecd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:36.204125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6431fd-0cc3-4719-b5a1-16a3a0c5ecd4 admin admin] PATH_INFO: `/v3/users/343192048b994ee39d3dbb1026f034d5` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:36.206874 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6431fd-0cc3-4719-b5a1-16a3a0c5ecd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=343192048b994ee39d3dbb1026f034d5)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:36.208963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6431fd-0cc3-4719-b5a1-16a3a0c5ecd4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:36.259985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3d6431fd-0cc3-4719-b5a1-16a3a0c5ecd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '343192048b994ee39d3dbb1026f034d5'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.272270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3d6431fd-0cc3-4719-b5a1-16a3a0c5ecd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '343192048b994ee39d3dbb1026f034d5'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.280665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3d6431fd-0cc3-4719-b5a1-16a3a0c5ecd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '343192048b994ee39d3dbb1026f034d5'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.281936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3d6431fd-0cc3-4719-b5a1-16a3a0c5ecd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '343192048b994ee39d3dbb1026f034d5'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.290453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3d6431fd-0cc3-4719-b5a1-16a3a0c5ecd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '343192048b994ee39d3dbb1026f034d5'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.296828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 938/1870] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:36 2022] DELETE /identity/v3/users/343192048b994ee39d3dbb1026f034d5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:36.759220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:36.761132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:36.774296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a1a05-85e2-4a92-9139-66509c2ff328 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:36.833153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d36ac-faaf-4190-ae02-d411608e4227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:36.834031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d36ac-faaf-4190-ae02-d411608e4227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:36.834367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d36ac-faaf-4190-ae02-d411608e4227 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:36.834591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d36ac-faaf-4190-ae02-d411608e4227 admin admin] PATH_INFO: `/v3/projects/781fae064b23446198039f4990070ebe` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:36.835892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d36ac-faaf-4190-ae02-d411608e4227 admin admin] RBAC: Authorizing `identity:delete_project(project_id=781fae064b23446198039f4990070ebe)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:36.837825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d36ac-faaf-4190-ae02-d411608e4227 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:36.884070 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-519d36ac-faaf-4190-ae02-d411608e4227 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '781fae064b23446198039f4990070ebe', 'request_id': 'req-519d36ac-faaf-4190-ae02-d411608e4227'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.891421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-519d36ac-faaf-4190-ae02-d411608e4227 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '781fae064b23446198039f4990070ebe', 'request_id': 'req-519d36ac-faaf-4190-ae02-d411608e4227'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.896953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-519d36ac-faaf-4190-ae02-d411608e4227 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '781fae064b23446198039f4990070ebe', 'request_id': 'req-519d36ac-faaf-4190-ae02-d411608e4227'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.908339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-519d36ac-faaf-4190-ae02-d411608e4227 admin admin] The token cache is being invalidate because project 781fae064b23446198039f4990070ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:36.908633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-519d36ac-faaf-4190-ae02-d411608e4227 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.909298 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-519d36ac-faaf-4190-ae02-d411608e4227 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:36.910869 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 933/1871] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:36 2022] DELETE /identity/v3/projects/781fae064b23446198039f4990070ebe => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:36.916362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217cffd-f2b2-407c-a530-140851f09e9b None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:36.916657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217cffd-f2b2-407c-a530-140851f09e9b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:36.916938 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217cffd-f2b2-407c-a530-140851f09e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:36.936109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-7217cffd-f2b2-407c-a530-140851f09e9b None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:36.937378 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7217cffd-f2b2-407c-a530-140851f09e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:37.015091 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 939/1872] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:37.017016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18f1b8-0d3a-47c6-98ec-bc993cbaf29f None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.017376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18f1b8-0d3a-47c6-98ec-bc993cbaf29f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.017725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18f1b8-0d3a-47c6-98ec-bc993cbaf29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.037137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-9e18f1b8-0d3a-47c6-98ec-bc993cbaf29f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:37.038350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9e18f1b8-0d3a-47c6-98ec-bc993cbaf29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:37.047566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 934/1873] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:37.051713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217cffd-f2b2-407c-a530-140851f09e9b None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:37.061481 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217cffd-f2b2-407c-a530-140851f09e9b None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:37.067065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd3ff0-1567-4fd8-a45d-cf5bc05151f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:37.067601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd3ff0-1567-4fd8-a45d-cf5bc05151f4 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.067895 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd3ff0-1567-4fd8-a45d-cf5bc05151f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.068210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd3ff0-1567-4fd8-a45d-cf5bc05151f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.068948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd3ff0-1567-4fd8-a45d-cf5bc05151f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:52:37.069262 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd3ff0-1567-4fd8-a45d-cf5bc05151f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:37.070885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd3ff0-1567-4fd8-a45d-cf5bc05151f4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:37.083008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 940/1874] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:52:37.087963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18f1b8-0d3a-47c6-98ec-bc993cbaf29f None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:37.097227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18f1b8-0d3a-47c6-98ec-bc993cbaf29f None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:37.101800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd4999-38b4-4297-9ab0-262795f7bbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:37.103021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd4999-38b4-4297-9ab0-262795f7bbbd admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.103307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd4999-38b4-4297-9ab0-262795f7bbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.103584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd4999-38b4-4297-9ab0-262795f7bbbd admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.104310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd4999-38b4-4297-9ab0-262795f7bbbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:37.106210 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd4999-38b4-4297-9ab0-262795f7bbbd admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:37.132186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 935/1875] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:37 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:37.136476 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd3ff0-1567-4fd8-a45d-cf5bc05151f4 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:37.146730 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd3ff0-1567-4fd8-a45d-cf5bc05151f4 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:37.188223 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:37.205257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d592a0-e582-43ac-91c0-f3d77ffbf3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:37.205871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d592a0-e582-43ac-91c0-f3d77ffbf3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.206161 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d592a0-e582-43ac-91c0-f3d77ffbf3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.206441 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d592a0-e582-43ac-91c0-f3d77ffbf3d0 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.207147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d592a0-e582-43ac-91c0-f3d77ffbf3d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:37.209094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d592a0-e582-43ac-91c0-f3d77ffbf3d0 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:37.231608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 941/1876] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:37 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:37.235634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd4999-38b4-4297-9ab0-262795f7bbbd admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:37.245690 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd4999-38b4-4297-9ab0-262795f7bbbd admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:37.250374 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b131c-b1b5-474e-99cc-be863c72fdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:37.251039 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b131c-b1b5-474e-99cc-be863c72fdb5 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.251316 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b131c-b1b5-474e-99cc-be863c72fdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.251591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b131c-b1b5-474e-99cc-be863c72fdb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.252347 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b131c-b1b5-474e-99cc-be863c72fdb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:37.253915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b131c-b1b5-474e-99cc-be863c72fdb5 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:37.264854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 936/1877] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:52:37.268778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d592a0-e582-43ac-91c0-f3d77ffbf3d0 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:37.279091 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d592a0-e582-43ac-91c0-f3d77ffbf3d0 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:37.283652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa65445-a2aa-4376-885e-452994919a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:37.284340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa65445-a2aa-4376-885e-452994919a3c admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.284624 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa65445-a2aa-4376-885e-452994919a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.284901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa65445-a2aa-4376-885e-452994919a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.285618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa65445-a2aa-4376-885e-452994919a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:37.287185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa65445-a2aa-4376-885e-452994919a3c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:37.298324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 942/1878] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:37.302419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b131c-b1b5-474e-99cc-be863c72fdb5 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:37.312151 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b131c-b1b5-474e-99cc-be863c72fdb5 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:37.317185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155ed2d-4af7-4e49-9619-165bb303cff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:37.318078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155ed2d-4af7-4e49-9619-165bb303cff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.318358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155ed2d-4af7-4e49-9619-165bb303cff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.318633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2155ed2d-4af7-4e49-9619-165bb303cff2 admin admin] PATH_INFO: `/v3/projects/802f7e4a5d0b4936aa97e04864dde1d4/users/2a4ef697d2b3454f84a60218559d8c24/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.332976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155ed2d-4af7-4e49-9619-165bb303cff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=802f7e4a5d0b4936aa97e04864dde1d4, user_id=2a4ef697d2b3454f84a60218559d8c24, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:37.335592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2155ed2d-4af7-4e49-9619-165bb303cff2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:37.349212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 937/1879] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:37 2022] PUT /identity/v3/projects/802f7e4a5d0b4936aa97e04864dde1d4/users/2a4ef697d2b3454f84a60218559d8c24/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:37.360674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b97123-7345-4814-8b19-87cf18e19a63 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.361132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b97123-7345-4814-8b19-87cf18e19a63 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.361422 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b97123-7345-4814-8b19-87cf18e19a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.391167 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f4b97123-7345-4814-8b19-87cf18e19a63 None None] MFA Rules not processed for user `2a4ef697d2b3454f84a60218559d8c24`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:37.392464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f4b97123-7345-4814-8b19-87cf18e19a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:37.460584 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 943/1880] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:37.465491 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155ed2d-4af7-4e49-9619-165bb303cff2 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:37.479817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:37.534889 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2155ed2d-4af7-4e49-9619-165bb303cff2 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:37.539733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e8960-249d-4d38-bd09-92b20be9b682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:37.540427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e8960-249d-4d38-bd09-92b20be9b682 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.540717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e8960-249d-4d38-bd09-92b20be9b682 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.540992 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305e8960-249d-4d38-bd09-92b20be9b682 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.541694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e8960-249d-4d38-bd09-92b20be9b682 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:37.543586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305e8960-249d-4d38-bd09-92b20be9b682 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:37.570749 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 938/1881] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:37 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:37.575443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b97123-7345-4814-8b19-87cf18e19a63 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:37.586024 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b97123-7345-4814-8b19-87cf18e19a63 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:37.658630 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadfcec-7254-4d72-bbc3-87f8faa18e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:37.659444 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadfcec-7254-4d72-bbc3-87f8faa18e64 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.659802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadfcec-7254-4d72-bbc3-87f8faa18e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.660192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadfcec-7254-4d72-bbc3-87f8faa18e64 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.660898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadfcec-7254-4d72-bbc3-87f8faa18e64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:37.663401 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadfcec-7254-4d72-bbc3-87f8faa18e64 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:37.691988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 944/1882] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:37 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:37.693463 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e8960-249d-4d38-bd09-92b20be9b682 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:37.705242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305e8960-249d-4d38-bd09-92b20be9b682 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:37.713109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a7f32-0980-4315-83a4-f73d396a6e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:37.713886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a7f32-0980-4315-83a4-f73d396a6e94 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.714268 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a7f32-0980-4315-83a4-f73d396a6e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.714623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a7f32-0980-4315-83a4-f73d396a6e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.715410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a7f32-0980-4315-83a4-f73d396a6e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:37.717057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096a7f32-0980-4315-83a4-f73d396a6e94 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:37.733126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 939/1883] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:37.739348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadfcec-7254-4d72-bbc3-87f8faa18e64 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:37.755045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadfcec-7254-4d72-bbc3-87f8faa18e64 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:37.761847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2761c97-5590-42e9-a62e-5ea2f2170c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:37.762817 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2761c97-5590-42e9-a62e-5ea2f2170c59 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.763341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2761c97-5590-42e9-a62e-5ea2f2170c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.763675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2761c97-5590-42e9-a62e-5ea2f2170c59 admin admin] PATH_INFO: `/v3/projects/bef452428d3d492a96beadb1b9d04067/users/ee6e2a31aeb64169917fff7cbfe277c0/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.787459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2761c97-5590-42e9-a62e-5ea2f2170c59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef452428d3d492a96beadb1b9d04067, user_id=ee6e2a31aeb64169917fff7cbfe277c0, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:37.790274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2761c97-5590-42e9-a62e-5ea2f2170c59 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:37.803973 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 945/1884] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:37 2022] PUT /identity/v3/projects/bef452428d3d492a96beadb1b9d04067/users/ee6e2a31aeb64169917fff7cbfe277c0/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:37.808600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a7f32-0980-4315-83a4-f73d396a6e94 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:37.818862 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a7f32-0980-4315-83a4-f73d396a6e94 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:37.895769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9882f7c-ae41-4b53-90ed-e2f9c68c3b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:37.896576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9882f7c-ae41-4b53-90ed-e2f9c68c3b40 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.896921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9882f7c-ae41-4b53-90ed-e2f9c68c3b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.897290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9882f7c-ae41-4b53-90ed-e2f9c68c3b40 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.898086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9882f7c-ae41-4b53-90ed-e2f9c68c3b40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:37.899798 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9882f7c-ae41-4b53-90ed-e2f9c68c3b40 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:37.909976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 940/1885] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:37 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:37.914183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2761c97-5590-42e9-a62e-5ea2f2170c59 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:37.924555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2761c97-5590-42e9-a62e-5ea2f2170c59 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:37.929305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c3727-3ca2-4ed5-a3a8-ce700ad1e5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:37.930196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c3727-3ca2-4ed5-a3a8-ce700ad1e5de admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.930971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c3727-3ca2-4ed5-a3a8-ce700ad1e5de admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.931282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3c3727-3ca2-4ed5-a3a8-ce700ad1e5de admin admin] PATH_INFO: `/v3/projects/bef452428d3d492a96beadb1b9d04067/users/ee6e2a31aeb64169917fff7cbfe277c0/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.933469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c3727-3ca2-4ed5-a3a8-ce700ad1e5de admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef452428d3d492a96beadb1b9d04067, user_id=ee6e2a31aeb64169917fff7cbfe277c0, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:37.936176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3c3727-3ca2-4ed5-a3a8-ce700ad1e5de admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:37.949331 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 946/1886] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:37 2022] PUT /identity/v3/projects/bef452428d3d492a96beadb1b9d04067/users/ee6e2a31aeb64169917fff7cbfe277c0/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:37.960285 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa068f8-9a7c-4b28-8647-5dd1d594f21c None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:37.960578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa068f8-9a7c-4b28-8647-5dd1d594f21c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:37.960861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa068f8-9a7c-4b28-8647-5dd1d594f21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:37.986216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5fa068f8-9a7c-4b28-8647-5dd1d594f21c None None] MFA Rules not processed for user `ee6e2a31aeb64169917fff7cbfe277c0`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:37.987412 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5fa068f8-9a7c-4b28-8647-5dd1d594f21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:38.053722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 941/1887] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:37 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:38.064966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c3727-3ca2-4ed5-a3a8-ce700ad1e5de admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:38.066645 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3d3c3727-3ca2-4ed5-a3a8-ce700ad1e5de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:38.073609 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:38.127963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3c3727-3ca2-4ed5-a3a8-ce700ad1e5de admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:38.131852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6ba94-7b92-414c-9b22-b3b5e8944b44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:38.132572 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6ba94-7b92-414c-9b22-b3b5e8944b44 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:38.132936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6ba94-7b92-414c-9b22-b3b5e8944b44 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:38.133224 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6ba94-7b92-414c-9b22-b3b5e8944b44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:38.143587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6ba94-7b92-414c-9b22-b3b5e8944b44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:38.145180 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6ba94-7b92-414c-9b22-b3b5e8944b44 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:38.156670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 947/1888] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:52:38.259438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa068f8-9a7c-4b28-8647-5dd1d594f21c None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:38.269191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa068f8-9a7c-4b28-8647-5dd1d594f21c None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:38.272998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3de6c-38be-4664-b567-f3595b6b8940 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:38.273674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3de6c-38be-4664-b567-f3595b6b8940 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:38.273919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3de6c-38be-4664-b567-f3595b6b8940 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:38.274240 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3de6c-38be-4664-b567-f3595b6b8940 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:38.278447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:38.331959 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3de6c-38be-4664-b567-f3595b6b8940 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:38.333317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3de6c-38be-4664-b567-f3595b6b8940 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:38.343195 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 942/1889] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:52:42.425513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6ba94-7b92-414c-9b22-b3b5e8944b44 service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:42.427431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-bce6ba94-7b92-414c-9b22-b3b5e8944b44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:42.440280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6ba94-7b92-414c-9b22-b3b5e8944b44 service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:42.445119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612d716-4ce1-4139-bd07-93c44b390d83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:42.445119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612d716-4ce1-4139-bd07-93c44b390d83 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:42.445119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612d716-4ce1-4139-bd07-93c44b390d83 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:42.445414 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612d716-4ce1-4139-bd07-93c44b390d83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:42.448735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-2612d716-4ce1-4139-bd07-93c44b390d83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:42.539235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612d716-4ce1-4139-bd07-93c44b390d83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:42.541044 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612d716-4ce1-4139-bd07-93c44b390d83 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:42.578758 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 948/1890] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:43.649067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafe5b0-cf29-4044-9fb6-320a13d28123 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:43.650228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafe5b0-cf29-4044-9fb6-320a13d28123 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:43.650228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbafe5b0-cf29-4044-9fb6-320a13d28123 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:43.651915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 943/1891] 10.222.0.64 () {54 vars in 921 bytes} [Thu Jul 14 17:52:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:52:43.672830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3c86d-99c0-4933-b05f-101d9fc0f8d7 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:43.673781 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3c86d-99c0-4933-b05f-101d9fc0f8d7 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:43.674200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3c86d-99c0-4933-b05f-101d9fc0f8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:43.696332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-21c3c86d-99c0-4933-b05f-101d9fc0f8d7 None None] MFA Rules not processed for user `6423f54152064b909d03c0908d636993`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:43.698258 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-21c3c86d-99c0-4933-b05f-101d9fc0f8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:43.806723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53948a2e-c363-4a3a-8b09-63c5ca04d8d9 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:43.807648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53948a2e-c363-4a3a-8b09-63c5ca04d8d9 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:43.807648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53948a2e-c363-4a3a-8b09-63c5ca04d8d9 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:43.809412 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 944/1892] 10.222.0.64 () {54 vars in 921 bytes} [Thu Jul 14 17:52:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:52:43.836486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 949/1893] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Jul 14 17:52:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:43.856143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3c86d-99c0-4933-b05f-101d9fc0f8d7 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:43.856643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86989600-629d-43d5-b277-f33b4fff482c None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:43.857137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86989600-629d-43d5-b277-f33b4fff482c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:43.857634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86989600-629d-43d5-b277-f33b4fff482c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:43.857739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-21c3c86d-99c0-4933-b05f-101d9fc0f8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:43.887191 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3c86d-99c0-4933-b05f-101d9fc0f8d7 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:43.903034 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-86989600-629d-43d5-b277-f33b4fff482c None None] MFA Rules not processed for user `6423f54152064b909d03c0908d636993`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:43.904309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-86989600-629d-43d5-b277-f33b4fff482c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:43.912849 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367a930-a59b-4e6d-a364-a5f75a682929 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:43.913606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367a930-a59b-4e6d-a364-a5f75a682929 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:43.913965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367a930-a59b-4e6d-a364-a5f75a682929 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:43.914239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367a930-a59b-4e6d-a364-a5f75a682929 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:43.932783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367a930-a59b-4e6d-a364-a5f75a682929 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:43.934422 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367a930-a59b-4e6d-a364-a5f75a682929 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:43.943119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 945/1894] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Jul 14 17:52:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:43.944379 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86989600-629d-43d5-b277-f33b4fff482c None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:43.947548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-86989600-629d-43d5-b277-f33b4fff482c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:43.956924 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 950/1895] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:44.035255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367a930-a59b-4e6d-a364-a5f75a682929 service nova] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:44.049524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367a930-a59b-4e6d-a364-a5f75a682929 service nova] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:44.067386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc69b35-5bf7-4ff0-9c13-90d234cda39a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:44.068375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc69b35-5bf7-4ff0-9c13-90d234cda39a service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:44.069379 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc69b35-5bf7-4ff0-9c13-90d234cda39a service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:44.070990 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc69b35-5bf7-4ff0-9c13-90d234cda39a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:44.083818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc69b35-5bf7-4ff0-9c13-90d234cda39a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:44.085443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86989600-629d-43d5-b277-f33b4fff482c None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:44.085999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc69b35-5bf7-4ff0-9c13-90d234cda39a service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:44.095996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c420a-f9f2-461c-9700-ab3f742ea601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:44.097180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c420a-f9f2-461c-9700-ab3f742ea601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:44.097537 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c420a-f9f2-461c-9700-ab3f742ea601 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:44.097882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c420a-f9f2-461c-9700-ab3f742ea601 admin admin] PATH_INFO: `/v3/users/c31dff105f89478194151a4450fde854` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:44.101965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c420a-f9f2-461c-9700-ab3f742ea601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c31dff105f89478194151a4450fde854)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:44.106383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c420a-f9f2-461c-9700-ab3f742ea601 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:44.120554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 951/1896] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:52:44.230894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f84c420a-f9f2-461c-9700-ab3f742ea601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c31dff105f89478194151a4450fde854'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:44.268852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f84c420a-f9f2-461c-9700-ab3f742ea601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c31dff105f89478194151a4450fde854'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:44.269746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f84c420a-f9f2-461c-9700-ab3f742ea601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c31dff105f89478194151a4450fde854'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:44.282655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f84c420a-f9f2-461c-9700-ab3f742ea601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c31dff105f89478194151a4450fde854'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:44.283536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f84c420a-f9f2-461c-9700-ab3f742ea601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c31dff105f89478194151a4450fde854'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:44.299101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 946/1897] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:43 2022] DELETE /identity/v3/users/c31dff105f89478194151a4450fde854 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:44.304028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd7914-4c71-4c54-977a-a5638468143d None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:44.304304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd7914-4c71-4c54-977a-a5638468143d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:44.304582 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd7914-4c71-4c54-977a-a5638468143d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:44.325816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:44.344328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-1ffd7914-4c71-4c54-977a-a5638468143d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:44.345559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1ffd7914-4c71-4c54-977a-a5638468143d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:44.444068 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 952/1898] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:44.451614 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c420a-f9f2-461c-9700-ab3f742ea601 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:44.453832 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f84c420a-f9f2-461c-9700-ab3f742ea601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:44.501653 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:44.552175 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c420a-f9f2-461c-9700-ab3f742ea601 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:44.556277 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a1dbf-686f-493a-81c8-ba279ebf36e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:44.557106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a1dbf-686f-493a-81c8-ba279ebf36e7 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:44.558095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a1dbf-686f-493a-81c8-ba279ebf36e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:44.558910 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4a1dbf-686f-493a-81c8-ba279ebf36e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:44.574498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a1dbf-686f-493a-81c8-ba279ebf36e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:44.576434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4a1dbf-686f-493a-81c8-ba279ebf36e7 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:44.599038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 947/1899] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:44.963207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd7914-4c71-4c54-977a-a5638468143d None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:44.964672 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1ffd7914-4c71-4c54-977a-a5638468143d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:44.977353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd7914-4c71-4c54-977a-a5638468143d None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:44.981993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:44.982870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:44.983156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:44.983437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] PATH_INFO: `/v3/projects/8c27532b39e645cfbfe67d989eb5aade` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:44.984856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c27532b39e645cfbfe67d989eb5aade)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:44.986851 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:45.034123 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c27532b39e645cfbfe67d989eb5aade', 'request_id': 'req-206e0ae1-e8f5-47d6-81a7-760ba82ac757'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.041445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c27532b39e645cfbfe67d989eb5aade', 'request_id': 'req-206e0ae1-e8f5-47d6-81a7-760ba82ac757'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.048043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c27532b39e645cfbfe67d989eb5aade', 'request_id': 'req-206e0ae1-e8f5-47d6-81a7-760ba82ac757'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.058755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] The token cache is being invalidate because project 8c27532b39e645cfbfe67d989eb5aade was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:45.059053 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.059742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.061383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 953/1900] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:44 2022] DELETE /identity/v3/projects/8c27532b39e645cfbfe67d989eb5aade => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:45.066294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a1dbf-686f-493a-81c8-ba279ebf36e7 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:45.076297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4a1dbf-686f-493a-81c8-ba279ebf36e7 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:45.132904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123eae3-b836-4a0b-9382-3ff077113eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:45.133788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123eae3-b836-4a0b-9382-3ff077113eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:45.134588 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123eae3-b836-4a0b-9382-3ff077113eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:45.134867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123eae3-b836-4a0b-9382-3ff077113eb8 admin admin] PATH_INFO: `/v3/users/9cdf7f997ff34a22ac1029c6ece4e419` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:45.136084 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123eae3-b836-4a0b-9382-3ff077113eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cdf7f997ff34a22ac1029c6ece4e419)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:45.138035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123eae3-b836-4a0b-9382-3ff077113eb8 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:45.183514 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a123eae3-b836-4a0b-9382-3ff077113eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cdf7f997ff34a22ac1029c6ece4e419'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.195863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a123eae3-b836-4a0b-9382-3ff077113eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cdf7f997ff34a22ac1029c6ece4e419'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.197065 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a123eae3-b836-4a0b-9382-3ff077113eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cdf7f997ff34a22ac1029c6ece4e419'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.208842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a123eae3-b836-4a0b-9382-3ff077113eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cdf7f997ff34a22ac1029c6ece4e419'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.209645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a123eae3-b836-4a0b-9382-3ff077113eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cdf7f997ff34a22ac1029c6ece4e419'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.222658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 948/1901] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:45 2022] DELETE /identity/v3/users/9cdf7f997ff34a22ac1029c6ece4e419 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:45.507761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:45.510083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:45.525058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206e0ae1-e8f5-47d6-81a7-760ba82ac757 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:45.608355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140510e-e730-4428-b642-3e39cf084722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:45.609421 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140510e-e730-4428-b642-3e39cf084722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:45.609800 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140510e-e730-4428-b642-3e39cf084722 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:45.610175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140510e-e730-4428-b642-3e39cf084722 admin admin] PATH_INFO: `/v3/projects/be858e8d9e8d4164bfae8802b5e4d199` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:45.611451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140510e-e730-4428-b642-3e39cf084722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be858e8d9e8d4164bfae8802b5e4d199)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:45.613520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140510e-e730-4428-b642-3e39cf084722 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:45.659735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:45.674239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7140510e-e730-4428-b642-3e39cf084722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be858e8d9e8d4164bfae8802b5e4d199', 'request_id': 'req-7140510e-e730-4428-b642-3e39cf084722'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.683166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7140510e-e730-4428-b642-3e39cf084722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be858e8d9e8d4164bfae8802b5e4d199', 'request_id': 'req-7140510e-e730-4428-b642-3e39cf084722'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.690348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7140510e-e730-4428-b642-3e39cf084722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be858e8d9e8d4164bfae8802b5e4d199', 'request_id': 'req-7140510e-e730-4428-b642-3e39cf084722'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.702314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7140510e-e730-4428-b642-3e39cf084722 admin admin] The token cache is being invalidate because project be858e8d9e8d4164bfae8802b5e4d199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:45.702605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7140510e-e730-4428-b642-3e39cf084722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.703267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7140510e-e730-4428-b642-3e39cf084722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:45.705012 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 954/1902] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:45 2022] DELETE /identity/v3/projects/be858e8d9e8d4164bfae8802b5e4d199 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:45.710317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e26fb-15ac-4069-a1e6-a21f980f598f None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:45.710613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e26fb-15ac-4069-a1e6-a21f980f598f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:45.710897 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740e26fb-15ac-4069-a1e6-a21f980f598f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:45.730736 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-740e26fb-15ac-4069-a1e6-a21f980f598f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:45.731972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-740e26fb-15ac-4069-a1e6-a21f980f598f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:45.770569 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:45.804870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 949/1903] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:45.808098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a8b37-5f17-443a-8ff2-b4fc7c7947e5 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:45.808565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a8b37-5f17-443a-8ff2-b4fc7c7947e5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:45.808850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a8b37-5f17-443a-8ff2-b4fc7c7947e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:45.828986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-e57a8b37-5f17-443a-8ff2-b4fc7c7947e5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:45.830256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e57a8b37-5f17-443a-8ff2-b4fc7c7947e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:45.839339 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 955/1904] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:45.843870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e26fb-15ac-4069-a1e6-a21f980f598f None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:45.855056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740e26fb-15ac-4069-a1e6-a21f980f598f None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:45.859949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c28f6a-9d5b-4525-9680-44eba28dbabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:45.860638 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c28f6a-9d5b-4525-9680-44eba28dbabf admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:45.860921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c28f6a-9d5b-4525-9680-44eba28dbabf admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:45.861197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c28f6a-9d5b-4525-9680-44eba28dbabf admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:45.861937 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c28f6a-9d5b-4525-9680-44eba28dbabf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:52:45.862274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c28f6a-9d5b-4525-9680-44eba28dbabf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:45.863858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c28f6a-9d5b-4525-9680-44eba28dbabf admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:45.876778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 950/1905] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:52:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:52:45.881539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a8b37-5f17-443a-8ff2-b4fc7c7947e5 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:45.890999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a8b37-5f17-443a-8ff2-b4fc7c7947e5 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:45.895934 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340ef68-d6d5-410a-9c33-e5f10b4609d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:45.896656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340ef68-d6d5-410a-9c33-e5f10b4609d7 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:45.896937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340ef68-d6d5-410a-9c33-e5f10b4609d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:45.897215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340ef68-d6d5-410a-9c33-e5f10b4609d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:45.897930 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340ef68-d6d5-410a-9c33-e5f10b4609d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:45.899891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340ef68-d6d5-410a-9c33-e5f10b4609d7 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:45.926019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 956/1906] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:45 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:52:45.930285 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c28f6a-9d5b-4525-9680-44eba28dbabf admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:45.941183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c28f6a-9d5b-4525-9680-44eba28dbabf admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:45.996607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107149f8-0064-446e-bd34-ac25d082926e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:45.997283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107149f8-0064-446e-bd34-ac25d082926e admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:45.997648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107149f8-0064-446e-bd34-ac25d082926e admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:45.997853 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107149f8-0064-446e-bd34-ac25d082926e admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:45.998630 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107149f8-0064-446e-bd34-ac25d082926e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:46.000490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107149f8-0064-446e-bd34-ac25d082926e admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:46.019661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 951/1907] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:45 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:46.023882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340ef68-d6d5-410a-9c33-e5f10b4609d7 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:46.034690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340ef68-d6d5-410a-9c33-e5f10b4609d7 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:46.039284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7f407-2979-4d9d-b8fc-44627ba9e00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:46.039982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7f407-2979-4d9d-b8fc-44627ba9e00c admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:46.040274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7f407-2979-4d9d-b8fc-44627ba9e00c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:46.040598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7f407-2979-4d9d-b8fc-44627ba9e00c admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:46.041812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7f407-2979-4d9d-b8fc-44627ba9e00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:46.043399 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7f407-2979-4d9d-b8fc-44627ba9e00c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:46.053948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 957/1908] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:46.058253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107149f8-0064-446e-bd34-ac25d082926e admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:46.069569 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107149f8-0064-446e-bd34-ac25d082926e admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:46.074310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4020de-c5a3-4660-b875-7c75751b3e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:46.075039 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4020de-c5a3-4660-b875-7c75751b3e02 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:46.075386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4020de-c5a3-4660-b875-7c75751b3e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:46.075675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4020de-c5a3-4660-b875-7c75751b3e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:46.076485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4020de-c5a3-4660-b875-7c75751b3e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:46.078051 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4020de-c5a3-4660-b875-7c75751b3e02 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:46.088844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 952/1909] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:46.092946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7f407-2979-4d9d-b8fc-44627ba9e00c admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:46.105330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7f407-2979-4d9d-b8fc-44627ba9e00c admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:46.110236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9446cbf-ebb9-48bd-9a20-c97e2baa60e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:46.111201 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9446cbf-ebb9-48bd-9a20-c97e2baa60e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:46.111428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9446cbf-ebb9-48bd-9a20-c97e2baa60e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:46.111709 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9446cbf-ebb9-48bd-9a20-c97e2baa60e9 admin admin] PATH_INFO: `/v3/projects/e97923a7855a49e98827ed3c1957a54a/users/546aeaa553dc41e8a4ed1262f4d12e4a/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:46.126869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9446cbf-ebb9-48bd-9a20-c97e2baa60e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e97923a7855a49e98827ed3c1957a54a, user_id=546aeaa553dc41e8a4ed1262f4d12e4a, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:46.129592 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9446cbf-ebb9-48bd-9a20-c97e2baa60e9 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:46.142800 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 958/1910] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:46 2022] PUT /identity/v3/projects/e97923a7855a49e98827ed3c1957a54a/users/546aeaa553dc41e8a4ed1262f4d12e4a/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:46.154628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822f8b2-f72e-4239-9e03-7621d65b013f None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:46.154923 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822f8b2-f72e-4239-9e03-7621d65b013f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:46.155203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822f8b2-f72e-4239-9e03-7621d65b013f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:46.182672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-7822f8b2-f72e-4239-9e03-7621d65b013f None None] MFA Rules not processed for user `546aeaa553dc41e8a4ed1262f4d12e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:46.183888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7822f8b2-f72e-4239-9e03-7621d65b013f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:46.246868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 953/1911] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:46 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:46.251472 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9446cbf-ebb9-48bd-9a20-c97e2baa60e9 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:46.313372 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9446cbf-ebb9-48bd-9a20-c97e2baa60e9 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:46.318418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a8a14-ca04-4f46-948c-eb61374a15bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:46.319082 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a8a14-ca04-4f46-948c-eb61374a15bb admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:46.319367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a8a14-ca04-4f46-948c-eb61374a15bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:46.319666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a8a14-ca04-4f46-948c-eb61374a15bb admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:46.320387 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a8a14-ca04-4f46-948c-eb61374a15bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:46.322365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a8a14-ca04-4f46-948c-eb61374a15bb admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:46.348280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 959/1912] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:46 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:46.352333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822f8b2-f72e-4239-9e03-7621d65b013f None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:46.363616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822f8b2-f72e-4239-9e03-7621d65b013f None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:46.401252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:46.429066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e9fdd-a5cf-4652-ab8a-80127242b86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:46.429858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e9fdd-a5cf-4652-ab8a-80127242b86a admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:46.430206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e9fdd-a5cf-4652-ab8a-80127242b86a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:46.430550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e9fdd-a5cf-4652-ab8a-80127242b86a admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:46.431389 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e9fdd-a5cf-4652-ab8a-80127242b86a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:46.433493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e9fdd-a5cf-4652-ab8a-80127242b86a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:46.459047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 954/1913] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:46 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:52:46.460407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a8a14-ca04-4f46-948c-eb61374a15bb admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:46.472382 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a8a14-ca04-4f46-948c-eb61374a15bb admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:46.477738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155abef-52b7-447c-ad0b-cfb16b073bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:46.478518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155abef-52b7-447c-ad0b-cfb16b073bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:46.478807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155abef-52b7-447c-ad0b-cfb16b073bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:46.479087 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b155abef-52b7-447c-ad0b-cfb16b073bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:46.479835 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b155abef-52b7-447c-ad0b-cfb16b073bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:46.481515 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b155abef-52b7-447c-ad0b-cfb16b073bb2 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:46.493541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 960/1914] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:46.497918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e9fdd-a5cf-4652-ab8a-80127242b86a admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:46.507767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e9fdd-a5cf-4652-ab8a-80127242b86a admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:46.513232 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807d6f1-4b4f-4a87-bb31-504eb16d543d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:46.514288 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807d6f1-4b4f-4a87-bb31-504eb16d543d admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:46.514577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807d6f1-4b4f-4a87-bb31-504eb16d543d admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:46.514858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807d6f1-4b4f-4a87-bb31-504eb16d543d admin admin] PATH_INFO: `/v3/projects/8aa9c60bae694c898aafb3b13d4e64e7/users/5b24696a2f43401c8171ac74cec4f7a6/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:46.531361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807d6f1-4b4f-4a87-bb31-504eb16d543d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aa9c60bae694c898aafb3b13d4e64e7, user_id=5b24696a2f43401c8171ac74cec4f7a6, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:46.534065 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807d6f1-4b4f-4a87-bb31-504eb16d543d admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:46.547459 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 955/1915] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:46 2022] PUT /identity/v3/projects/8aa9c60bae694c898aafb3b13d4e64e7/users/5b24696a2f43401c8171ac74cec4f7a6/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:46.551112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155abef-52b7-447c-ad0b-cfb16b073bb2 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:46.561208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b155abef-52b7-447c-ad0b-cfb16b073bb2 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:46.593585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:46.623725 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18e5d3-7469-4c7e-a9bb-7b790338ec13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:46.624436 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18e5d3-7469-4c7e-a9bb-7b790338ec13 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:46.624723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18e5d3-7469-4c7e-a9bb-7b790338ec13 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:46.625002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18e5d3-7469-4c7e-a9bb-7b790338ec13 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:46.625726 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18e5d3-7469-4c7e-a9bb-7b790338ec13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:46.627379 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18e5d3-7469-4c7e-a9bb-7b790338ec13 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:46.641204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 961/1916] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:46 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:52:46.642529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807d6f1-4b4f-4a87-bb31-504eb16d543d admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:46.653724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807d6f1-4b4f-4a87-bb31-504eb16d543d admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:46.658755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460338f4-793d-4ac0-bc10-7aae9cc4ff85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:46.659378 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460338f4-793d-4ac0-bc10-7aae9cc4ff85 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:46.659963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460338f4-793d-4ac0-bc10-7aae9cc4ff85 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:46.660079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460338f4-793d-4ac0-bc10-7aae9cc4ff85 admin admin] PATH_INFO: `/v3/projects/8aa9c60bae694c898aafb3b13d4e64e7/users/5b24696a2f43401c8171ac74cec4f7a6/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:46.662642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460338f4-793d-4ac0-bc10-7aae9cc4ff85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aa9c60bae694c898aafb3b13d4e64e7, user_id=5b24696a2f43401c8171ac74cec4f7a6, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:46.665533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460338f4-793d-4ac0-bc10-7aae9cc4ff85 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:46.677629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 956/1917] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:46 2022] PUT /identity/v3/projects/8aa9c60bae694c898aafb3b13d4e64e7/users/5b24696a2f43401c8171ac74cec4f7a6/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:46.688257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18521b-79ff-4524-9cda-148553293902 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:46.688558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18521b-79ff-4524-9cda-148553293902 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:46.688822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18521b-79ff-4524-9cda-148553293902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:46.717937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-ea18521b-79ff-4524-9cda-148553293902 None None] MFA Rules not processed for user `5b24696a2f43401c8171ac74cec4f7a6`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:46.719194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ea18521b-79ff-4524-9cda-148553293902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:46.807460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 962/1918] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:46 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:46.819198 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460338f4-793d-4ac0-bc10-7aae9cc4ff85 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:46.821350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-460338f4-793d-4ac0-bc10-7aae9cc4ff85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:46.897004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460338f4-793d-4ac0-bc10-7aae9cc4ff85 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:46.901245 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63851f3-9d03-4db1-8f73-cc886445434d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:46.901915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63851f3-9d03-4db1-8f73-cc886445434d service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:46.902195 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63851f3-9d03-4db1-8f73-cc886445434d service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:46.902468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63851f3-9d03-4db1-8f73-cc886445434d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:46.914061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63851f3-9d03-4db1-8f73-cc886445434d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:46.916181 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63851f3-9d03-4db1-8f73-cc886445434d service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:46.930964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 957/1919] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:46 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:46.976622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18521b-79ff-4524-9cda-148553293902 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:46.978910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ea18521b-79ff-4524-9cda-148553293902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:46.991943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18521b-79ff-4524-9cda-148553293902 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:46.995665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed563bbd-86e5-4963-a6e7-3c39daf05e7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:46.996490 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed563bbd-86e5-4963-a6e7-3c39daf05e7b service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:46.996776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed563bbd-86e5-4963-a6e7-3c39daf05e7b service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:46.997056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed563bbd-86e5-4963-a6e7-3c39daf05e7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:47.061229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed563bbd-86e5-4963-a6e7-3c39daf05e7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:47.062817 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed563bbd-86e5-4963-a6e7-3c39daf05e7b service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:47.073661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 963/1920] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:46 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:48.627623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63851f3-9d03-4db1-8f73-cc886445434d service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:48.629648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f63851f3-9d03-4db1-8f73-cc886445434d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:48.717692 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63851f3-9d03-4db1-8f73-cc886445434d service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:48.722586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9015a-d4fe-4ea3-9f58-8ed7b44921ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:48.723590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9015a-d4fe-4ea3-9f58-8ed7b44921ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:48.723988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9015a-d4fe-4ea3-9f58-8ed7b44921ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:48.724334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e9015a-d4fe-4ea3-9f58-8ed7b44921ee admin admin] PATH_INFO: `/v3/users/2a4ef697d2b3454f84a60218559d8c24` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:48.726096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9015a-d4fe-4ea3-9f58-8ed7b44921ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a4ef697d2b3454f84a60218559d8c24)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:48.728279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e9015a-d4fe-4ea3-9f58-8ed7b44921ee admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:48.763196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:48.780523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-96e9015a-d4fe-4ea3-9f58-8ed7b44921ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a4ef697d2b3454f84a60218559d8c24'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:48.798469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-96e9015a-d4fe-4ea3-9f58-8ed7b44921ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a4ef697d2b3454f84a60218559d8c24'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:48.799216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-96e9015a-d4fe-4ea3-9f58-8ed7b44921ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a4ef697d2b3454f84a60218559d8c24'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:48.807241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-96e9015a-d4fe-4ea3-9f58-8ed7b44921ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a4ef697d2b3454f84a60218559d8c24'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:48.807994 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-96e9015a-d4fe-4ea3-9f58-8ed7b44921ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a4ef697d2b3454f84a60218559d8c24'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:48.824843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 958/1921] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:48 2022] DELETE /identity/v3/users/2a4ef697d2b3454f84a60218559d8c24 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:48.830066 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189a33b-4c6b-4621-b8fe-66950fd853bb None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:48.830693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189a33b-4c6b-4621-b8fe-66950fd853bb None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:48.830693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189a33b-4c6b-4621-b8fe-66950fd853bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:48.852119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-1189a33b-4c6b-4621-b8fe-66950fd853bb None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:48.854500 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1189a33b-4c6b-4621-b8fe-66950fd853bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:48.917919 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:48.938333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 964/1922] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:48.958594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9015a-d4fe-4ea3-9f58-8ed7b44921ee admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:48.960509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-96e9015a-d4fe-4ea3-9f58-8ed7b44921ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:49.029304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e9015a-d4fe-4ea3-9f58-8ed7b44921ee admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:49.032870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447aba0c-c975-445a-b237-ffae6918ed1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:49.033529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aba0c-c975-445a-b237-ffae6918ed1d service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:49.033814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aba0c-c975-445a-b237-ffae6918ed1d service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:49.034085 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aba0c-c975-445a-b237-ffae6918ed1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:49.044743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447aba0c-c975-445a-b237-ffae6918ed1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:49.046855 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447aba0c-c975-445a-b237-ffae6918ed1d service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:49.058978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 959/1923] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:49.390531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189a33b-4c6b-4621-b8fe-66950fd853bb None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:49.393254 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1189a33b-4c6b-4621-b8fe-66950fd853bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:49.420582 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189a33b-4c6b-4621-b8fe-66950fd853bb None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:49.430296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:49.431790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:49.432516 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:49.432812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] PATH_INFO: `/v3/projects/802f7e4a5d0b4936aa97e04864dde1d4` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:49.435094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=802f7e4a5d0b4936aa97e04864dde1d4)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:49.439709 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:49.524254 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '802f7e4a5d0b4936aa97e04864dde1d4', 'request_id': 'req-96df618c-61da-4b1f-be69-4570b2d56ad4'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:49.533766 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '802f7e4a5d0b4936aa97e04864dde1d4', 'request_id': 'req-96df618c-61da-4b1f-be69-4570b2d56ad4'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:49.540982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '802f7e4a5d0b4936aa97e04864dde1d4', 'request_id': 'req-96df618c-61da-4b1f-be69-4570b2d56ad4'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:49.553105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] The token cache is being invalidate because project 802f7e4a5d0b4936aa97e04864dde1d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:49.553617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:49.554386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:49.556428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 965/1924] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:49 2022] DELETE /identity/v3/projects/802f7e4a5d0b4936aa97e04864dde1d4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:49.560465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447aba0c-c975-445a-b237-ffae6918ed1d service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:49.573875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447aba0c-c975-445a-b237-ffae6918ed1d service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:49.634237 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c662a14-a5c5-4ffa-b056-90df63114a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:49.636386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c662a14-a5c5-4ffa-b056-90df63114a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:49.636673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c662a14-a5c5-4ffa-b056-90df63114a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:49.636953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c662a14-a5c5-4ffa-b056-90df63114a27 admin admin] PATH_INFO: `/v3/users/ee6e2a31aeb64169917fff7cbfe277c0` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:49.638205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c662a14-a5c5-4ffa-b056-90df63114a27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee6e2a31aeb64169917fff7cbfe277c0)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:49.640170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c662a14-a5c5-4ffa-b056-90df63114a27 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:49.699484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1c662a14-a5c5-4ffa-b056-90df63114a27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee6e2a31aeb64169917fff7cbfe277c0'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:49.706573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:49.720103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1c662a14-a5c5-4ffa-b056-90df63114a27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee6e2a31aeb64169917fff7cbfe277c0'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:49.721162 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1c662a14-a5c5-4ffa-b056-90df63114a27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee6e2a31aeb64169917fff7cbfe277c0'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:49.730692 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1c662a14-a5c5-4ffa-b056-90df63114a27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee6e2a31aeb64169917fff7cbfe277c0'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:49.731324 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1c662a14-a5c5-4ffa-b056-90df63114a27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee6e2a31aeb64169917fff7cbfe277c0'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:49.745216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 960/1925] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:49 2022] DELETE /identity/v3/users/ee6e2a31aeb64169917fff7cbfe277c0 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:50.081557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:50.082961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:50.106681 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df618c-61da-4b1f-be69-4570b2d56ad4 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:50.179555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb714f2-680e-4266-9912-2848fe66715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:50.180471 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb714f2-680e-4266-9912-2848fe66715a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:50.180750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb714f2-680e-4266-9912-2848fe66715a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:50.181087 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb714f2-680e-4266-9912-2848fe66715a admin admin] PATH_INFO: `/v3/projects/bef452428d3d492a96beadb1b9d04067` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:50.182853 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb714f2-680e-4266-9912-2848fe66715a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bef452428d3d492a96beadb1b9d04067)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:50.184794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb714f2-680e-4266-9912-2848fe66715a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:50.282755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-6bb714f2-680e-4266-9912-2848fe66715a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bef452428d3d492a96beadb1b9d04067', 'request_id': 'req-6bb714f2-680e-4266-9912-2848fe66715a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:50.302487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-6bb714f2-680e-4266-9912-2848fe66715a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bef452428d3d492a96beadb1b9d04067', 'request_id': 'req-6bb714f2-680e-4266-9912-2848fe66715a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:50.324548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-6bb714f2-680e-4266-9912-2848fe66715a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bef452428d3d492a96beadb1b9d04067', 'request_id': 'req-6bb714f2-680e-4266-9912-2848fe66715a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:50.346928 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-6bb714f2-680e-4266-9912-2848fe66715a admin admin] The token cache is being invalidate because project bef452428d3d492a96beadb1b9d04067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:50.347218 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-6bb714f2-680e-4266-9912-2848fe66715a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:50.347889 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-6bb714f2-680e-4266-9912-2848fe66715a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:50.354867 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 966/1926] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:50 2022] DELETE /identity/v3/projects/bef452428d3d492a96beadb1b9d04067 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:50.356417 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e4d84-14b9-4312-920b-026f1ff1d130 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:50.356713 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e4d84-14b9-4312-920b-026f1ff1d130 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:50.357042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e4d84-14b9-4312-920b-026f1ff1d130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:50.387388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-168e4d84-14b9-4312-920b-026f1ff1d130 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:50.390433 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-168e4d84-14b9-4312-920b-026f1ff1d130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:50.499979 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 961/1927] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:52:50.504456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de4e09-b379-468f-b534-4d5bf61586ab None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:50.504783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de4e09-b379-468f-b534-4d5bf61586ab None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:50.505045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de4e09-b379-468f-b534-4d5bf61586ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:50.512964 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:50.529168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-01de4e09-b379-468f-b534-4d5bf61586ab None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:50.531325 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-01de4e09-b379-468f-b534-4d5bf61586ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:50.556434 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 967/1928] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:50.561325 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e4d84-14b9-4312-920b-026f1ff1d130 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:50.572586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e4d84-14b9-4312-920b-026f1ff1d130 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:50.578344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011b5c2-6968-495e-a762-0dc94a14e90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:50.579186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011b5c2-6968-495e-a762-0dc94a14e90f admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:50.579538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011b5c2-6968-495e-a762-0dc94a14e90f admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:50.579880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8011b5c2-6968-495e-a762-0dc94a14e90f admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:50.580719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011b5c2-6968-495e-a762-0dc94a14e90f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:52:50.581093 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011b5c2-6968-495e-a762-0dc94a14e90f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:50.582753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8011b5c2-6968-495e-a762-0dc94a14e90f admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:50.596659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 962/1929] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:52:50.601312 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de4e09-b379-468f-b534-4d5bf61586ab None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:50.612614 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de4e09-b379-468f-b534-4d5bf61586ab None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:50.617591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b7361-0d85-4a5e-a90f-bfb4d8a1f671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:50.618265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b7361-0d85-4a5e-a90f-bfb4d8a1f671 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:50.618545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b7361-0d85-4a5e-a90f-bfb4d8a1f671 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:50.618871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b7361-0d85-4a5e-a90f-bfb4d8a1f671 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:50.619576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b7361-0d85-4a5e-a90f-bfb4d8a1f671 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:50.621791 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b7361-0d85-4a5e-a90f-bfb4d8a1f671 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:50.650752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 968/1930] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:50 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:50.655070 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011b5c2-6968-495e-a762-0dc94a14e90f admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:50.665391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8011b5c2-6968-495e-a762-0dc94a14e90f admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:50.726979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b7361-0d85-4a5e-a90f-bfb4d8a1f671 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:50.731164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-570b7361-0d85-4a5e-a90f-bfb4d8a1f671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:50.761890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996fe84-6b08-4647-a1a6-3adebccdf8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:50.763053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996fe84-6b08-4647-a1a6-3adebccdf8fd admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:50.763346 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996fe84-6b08-4647-a1a6-3adebccdf8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:50.763629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996fe84-6b08-4647-a1a6-3adebccdf8fd admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:50.764945 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996fe84-6b08-4647-a1a6-3adebccdf8fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:50.767231 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996fe84-6b08-4647-a1a6-3adebccdf8fd admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:50.791109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 963/1931] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:50 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:50.794900 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996fe84-6b08-4647-a1a6-3adebccdf8fd admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:50.810025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b7361-0d85-4a5e-a90f-bfb4d8a1f671 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:50.810789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996fe84-6b08-4647-a1a6-3adebccdf8fd admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:50.815086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57267e-7bb2-4c36-9d99-944f17e5a604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:50.816167 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57267e-7bb2-4c36-9d99-944f17e5a604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:50.816453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57267e-7bb2-4c36-9d99-944f17e5a604 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:50.816733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57267e-7bb2-4c36-9d99-944f17e5a604 admin admin] PATH_INFO: `/v3/users/546aeaa553dc41e8a4ed1262f4d12e4a` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:50.816837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1c205-7c2e-498f-8b32-da3b2aa2950c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:50.817483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1c205-7c2e-498f-8b32-da3b2aa2950c admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:50.817777 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1c205-7c2e-498f-8b32-da3b2aa2950c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:50.818064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1c205-7c2e-498f-8b32-da3b2aa2950c admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:50.818705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57267e-7bb2-4c36-9d99-944f17e5a604 admin admin] RBAC: Authorizing `identity:delete_user(user_id=546aeaa553dc41e8a4ed1262f4d12e4a)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:50.818927 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1c205-7c2e-498f-8b32-da3b2aa2950c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:50.820675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57267e-7bb2-4c36-9d99-944f17e5a604 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:50.820922 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1c205-7c2e-498f-8b32-da3b2aa2950c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:50.840327 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 964/1932] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:52:50.842685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1c205-7c2e-498f-8b32-da3b2aa2950c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:50.856438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1c205-7c2e-498f-8b32-da3b2aa2950c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:50.869140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59326cad-8daa-46bc-a13b-021c3dbb88f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:50.869821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59326cad-8daa-46bc-a13b-021c3dbb88f6 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:50.870106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59326cad-8daa-46bc-a13b-021c3dbb88f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:50.870380 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59326cad-8daa-46bc-a13b-021c3dbb88f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:50.871114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59326cad-8daa-46bc-a13b-021c3dbb88f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:50.872738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59326cad-8daa-46bc-a13b-021c3dbb88f6 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:50.884649 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 965/1933] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:52:50.885191 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-bf57267e-7bb2-4c36-9d99-944f17e5a604 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '546aeaa553dc41e8a4ed1262f4d12e4a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:50.889962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59326cad-8daa-46bc-a13b-021c3dbb88f6 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:50.898688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-bf57267e-7bb2-4c36-9d99-944f17e5a604 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '546aeaa553dc41e8a4ed1262f4d12e4a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:50.899675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59326cad-8daa-46bc-a13b-021c3dbb88f6 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:50.905506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39108a6f-94e4-4262-a652-f65dc0d3c5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:50.906422 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39108a6f-94e4-4262-a652-f65dc0d3c5f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:50.906703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39108a6f-94e4-4262-a652-f65dc0d3c5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:50.906982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39108a6f-94e4-4262-a652-f65dc0d3c5f9 admin admin] PATH_INFO: `/v3/projects/2d87977f834942e588078bcd16f21a2e/users/c3927e88892e45479489cd3910ee35bd/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:50.907648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-bf57267e-7bb2-4c36-9d99-944f17e5a604 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '546aeaa553dc41e8a4ed1262f4d12e4a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:50.908345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-bf57267e-7bb2-4c36-9d99-944f17e5a604 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '546aeaa553dc41e8a4ed1262f4d12e4a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:50.916652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-bf57267e-7bb2-4c36-9d99-944f17e5a604 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '546aeaa553dc41e8a4ed1262f4d12e4a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:50.922586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39108a6f-94e4-4262-a652-f65dc0d3c5f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d87977f834942e588078bcd16f21a2e, user_id=c3927e88892e45479489cd3910ee35bd, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:50.924806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 969/1934] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:50 2022] DELETE /identity/v3/users/546aeaa553dc41e8a4ed1262f4d12e4a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:52:50.927657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39108a6f-94e4-4262-a652-f65dc0d3c5f9 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:50.929058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25f835-c4a2-4e07-878f-512aad2f6518 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:50.929354 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25f835-c4a2-4e07-878f-512aad2f6518 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:50.929659 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df25f835-c4a2-4e07-878f-512aad2f6518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:50.939886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 966/1935] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:50 2022] PUT /identity/v3/projects/2d87977f834942e588078bcd16f21a2e/users/c3927e88892e45479489cd3910ee35bd/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:52:50.950601 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a147f4-7519-4839-855c-7ac51baae3b3 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:50.950892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a147f4-7519-4839-855c-7ac51baae3b3 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:50.951308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a147f4-7519-4839-855c-7ac51baae3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:50.952386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-df25f835-c4a2-4e07-878f-512aad2f6518 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:50.954248 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-df25f835-c4a2-4e07-878f-512aad2f6518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:50.957001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:50.980019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-d7a147f4-7519-4839-855c-7ac51baae3b3 None None] MFA Rules not processed for user `c3927e88892e45479489cd3910ee35bd`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:50.981219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d7a147f4-7519-4839-855c-7ac51baae3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:51.011511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:51.023030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 970/1936] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:52:51.033094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25f835-c4a2-4e07-878f-512aad2f6518 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:51.034453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-df25f835-c4a2-4e07-878f-512aad2f6518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:51.053211 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 967/1937] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:52:51.065180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a147f4-7519-4839-855c-7ac51baae3b3 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:51.066556 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d7a147f4-7519-4839-855c-7ac51baae3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:51.091791 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df25f835-c4a2-4e07-878f-512aad2f6518 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:51.096140 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9821a0b-57e3-4b9d-b7dd-e095f18868c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:51.096803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821a0b-57e3-4b9d-b7dd-e095f18868c1 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:51.097083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821a0b-57e3-4b9d-b7dd-e095f18868c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:51.097358 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821a0b-57e3-4b9d-b7dd-e095f18868c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:51.107234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9821a0b-57e3-4b9d-b7dd-e095f18868c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:51.108867 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9821a0b-57e3-4b9d-b7dd-e095f18868c1 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:51.118939 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 971/1938] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:52:51.134132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a147f4-7519-4839-855c-7ac51baae3b3 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:51.138364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e733b9fb-f890-40ca-9b7f-3b2119306f6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:51.139027 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733b9fb-f890-40ca-9b7f-3b2119306f6d service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:51.139311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733b9fb-f890-40ca-9b7f-3b2119306f6d service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:51.139584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e733b9fb-f890-40ca-9b7f-3b2119306f6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:51.148404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733b9fb-f890-40ca-9b7f-3b2119306f6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:51.150693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e733b9fb-f890-40ca-9b7f-3b2119306f6d service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:51.161132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 968/1939] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:52:51.182043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9821a0b-57e3-4b9d-b7dd-e095f18868c1 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:51.183585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c9821a0b-57e3-4b9d-b7dd-e095f18868c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:51.197532 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9821a0b-57e3-4b9d-b7dd-e095f18868c1 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:51.203077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a91722-835c-40da-a4db-523f8329805b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:51.203077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a91722-835c-40da-a4db-523f8329805b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:51.203760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a91722-835c-40da-a4db-523f8329805b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:51.203980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a91722-835c-40da-a4db-523f8329805b admin admin] PATH_INFO: `/v3/users/c3927e88892e45479489cd3910ee35bd` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:51.205336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a91722-835c-40da-a4db-523f8329805b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3927e88892e45479489cd3910ee35bd)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:51.207135 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a91722-835c-40da-a4db-523f8329805b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:51.256723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-47a91722-835c-40da-a4db-523f8329805b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3927e88892e45479489cd3910ee35bd'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.269933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-47a91722-835c-40da-a4db-523f8329805b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3927e88892e45479489cd3910ee35bd'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.282741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-47a91722-835c-40da-a4db-523f8329805b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3927e88892e45479489cd3910ee35bd'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.283413 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-47a91722-835c-40da-a4db-523f8329805b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3927e88892e45479489cd3910ee35bd'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.291315 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-47a91722-835c-40da-a4db-523f8329805b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3927e88892e45479489cd3910ee35bd'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.297648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 972/1940] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:51 2022] DELETE /identity/v3/users/c3927e88892e45479489cd3910ee35bd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:51.302096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a4083-3a3b-4340-a88b-2afafe95cb3d None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:51.302892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a4083-3a3b-4340-a88b-2afafe95cb3d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:51.302892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a4083-3a3b-4340-a88b-2afafe95cb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:51.321193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4d7a4083-3a3b-4340-a88b-2afafe95cb3d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:51.323106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4d7a4083-3a3b-4340-a88b-2afafe95cb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:51.385389 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 969/1941] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:51.395790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a91722-835c-40da-a4db-523f8329805b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:51.397393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-47a91722-835c-40da-a4db-523f8329805b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:51.408449 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a4083-3a3b-4340-a88b-2afafe95cb3d None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:51.409889 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4d7a4083-3a3b-4340-a88b-2afafe95cb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:51.421952 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a4083-3a3b-4340-a88b-2afafe95cb3d None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:51.428388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:51.429474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:51.429928 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:51.430364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] PATH_INFO: `/v3/projects/e97923a7855a49e98827ed3c1957a54a` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:51.432378 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e97923a7855a49e98827ed3c1957a54a)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:51.434756 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:51.487705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a91722-835c-40da-a4db-523f8329805b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:51.488618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:51.505163 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e97923a7855a49e98827ed3c1957a54a', 'request_id': 'req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.517537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:51.520008 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e97923a7855a49e98827ed3c1957a54a', 'request_id': 'req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.528966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e97923a7855a49e98827ed3c1957a54a', 'request_id': 'req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.541773 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] The token cache is being invalidate because project e97923a7855a49e98827ed3c1957a54a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:51.542070 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.542801 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.544562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 970/1942] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:51 2022] DELETE /identity/v3/projects/e97923a7855a49e98827ed3c1957a54a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:52:51.548836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:51.550441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:51.553941 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062a0c1-5ad3-4d17-bdc2-ec844b10452a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:51.554664 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062a0c1-5ad3-4d17-bdc2-ec844b10452a service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:51.556080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062a0c1-5ad3-4d17-bdc2-ec844b10452a service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:51.556080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062a0c1-5ad3-4d17-bdc2-ec844b10452a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:51.557138 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-2062a0c1-5ad3-4d17-bdc2-ec844b10452a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:51.651702 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed7abf-1a66-46e2-845f-efe3fd4c0f2e admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:51.654048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062a0c1-5ad3-4d17-bdc2-ec844b10452a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:51.655975 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c12f0-a7df-432f-89d0-4b3c31346dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:51.656853 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c12f0-a7df-432f-89d0-4b3c31346dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:51.657137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c12f0-a7df-432f-89d0-4b3c31346dab admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:51.657417 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c12f0-a7df-432f-89d0-4b3c31346dab admin admin] PATH_INFO: `/v3/users/5b24696a2f43401c8171ac74cec4f7a6` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:51.658327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2062a0c1-5ad3-4d17-bdc2-ec844b10452a service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:51.658682 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c12f0-a7df-432f-89d0-4b3c31346dab admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b24696a2f43401c8171ac74cec4f7a6)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:51.660661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c12f0-a7df-432f-89d0-4b3c31346dab admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:51.672032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 973/1943] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:51.715565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-134c12f0-a7df-432f-89d0-4b3c31346dab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b24696a2f43401c8171ac74cec4f7a6'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.733118 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-134c12f0-a7df-432f-89d0-4b3c31346dab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b24696a2f43401c8171ac74cec4f7a6'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.733939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-134c12f0-a7df-432f-89d0-4b3c31346dab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b24696a2f43401c8171ac74cec4f7a6'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.743679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-134c12f0-a7df-432f-89d0-4b3c31346dab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b24696a2f43401c8171ac74cec4f7a6'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.745217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-134c12f0-a7df-432f-89d0-4b3c31346dab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b24696a2f43401c8171ac74cec4f7a6'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:51.763918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 971/1944] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:51 2022] DELETE /identity/v3/users/5b24696a2f43401c8171ac74cec4f7a6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:52.186982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062a0c1-5ad3-4d17-bdc2-ec844b10452a service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:52.203294 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062a0c1-5ad3-4d17-bdc2-ec844b10452a service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:52.241307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c12f0-a7df-432f-89d0-4b3c31346dab admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:52.243984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-134c12f0-a7df-432f-89d0-4b3c31346dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:52.302488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc479f-ee55-41a5-8527-ecf19c77fc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:52.303381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc479f-ee55-41a5-8527-ecf19c77fc64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.303665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc479f-ee55-41a5-8527-ecf19c77fc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.303970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc479f-ee55-41a5-8527-ecf19c77fc64 admin admin] PATH_INFO: `/v3/projects/8aa9c60bae694c898aafb3b13d4e64e7` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.305242 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc479f-ee55-41a5-8527-ecf19c77fc64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aa9c60bae694c898aafb3b13d4e64e7)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:52.305741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:52.307960 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc479f-ee55-41a5-8527-ecf19c77fc64 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:52.334428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c12f0-a7df-432f-89d0-4b3c31346dab admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:52.339883 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a576788e-97d8-4598-b075-d1bc9dd433a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:52.340987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576788e-97d8-4598-b075-d1bc9dd433a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.341810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576788e-97d8-4598-b075-d1bc9dd433a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.341810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576788e-97d8-4598-b075-d1bc9dd433a0 admin admin] PATH_INFO: `/v3/projects/2d87977f834942e588078bcd16f21a2e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.343093 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576788e-97d8-4598-b075-d1bc9dd433a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d87977f834942e588078bcd16f21a2e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:52.345205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a576788e-97d8-4598-b075-d1bc9dd433a0 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:52.363608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d2bc479f-ee55-41a5-8527-ecf19c77fc64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aa9c60bae694c898aafb3b13d4e64e7', 'request_id': 'req-d2bc479f-ee55-41a5-8527-ecf19c77fc64'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:52.371731 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d2bc479f-ee55-41a5-8527-ecf19c77fc64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aa9c60bae694c898aafb3b13d4e64e7', 'request_id': 'req-d2bc479f-ee55-41a5-8527-ecf19c77fc64'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:52.378453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d2bc479f-ee55-41a5-8527-ecf19c77fc64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aa9c60bae694c898aafb3b13d4e64e7', 'request_id': 'req-d2bc479f-ee55-41a5-8527-ecf19c77fc64'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:52.389666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d2bc479f-ee55-41a5-8527-ecf19c77fc64 admin admin] The token cache is being invalidate because project 8aa9c60bae694c898aafb3b13d4e64e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:52.389955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d2bc479f-ee55-41a5-8527-ecf19c77fc64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:52.390597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d2bc479f-ee55-41a5-8527-ecf19c77fc64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:52.392389 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 974/1945] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:52 2022] DELETE /identity/v3/projects/8aa9c60bae694c898aafb3b13d4e64e7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:52.396140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a576788e-97d8-4598-b075-d1bc9dd433a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d87977f834942e588078bcd16f21a2e', 'request_id': 'req-a576788e-97d8-4598-b075-d1bc9dd433a0'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:52.396955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8921e-af4b-4d82-aa1f-b5464c20755b None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.397242 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8921e-af4b-4d82-aa1f-b5464c20755b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.397520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a8921e-af4b-4d82-aa1f-b5464c20755b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.408500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a576788e-97d8-4598-b075-d1bc9dd433a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d87977f834942e588078bcd16f21a2e', 'request_id': 'req-a576788e-97d8-4598-b075-d1bc9dd433a0'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:52.410474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:52.418040 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a576788e-97d8-4598-b075-d1bc9dd433a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d87977f834942e588078bcd16f21a2e', 'request_id': 'req-a576788e-97d8-4598-b075-d1bc9dd433a0'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:52.418232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-c6a8921e-af4b-4d82-aa1f-b5464c20755b None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:52.419309 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c6a8921e-af4b-4d82-aa1f-b5464c20755b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:52.433941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a576788e-97d8-4598-b075-d1bc9dd433a0 admin admin] The token cache is being invalidate because project 2d87977f834942e588078bcd16f21a2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:52.434325 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a576788e-97d8-4598-b075-d1bc9dd433a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:52.435074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a576788e-97d8-4598-b075-d1bc9dd433a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:52.436890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 972/1946] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:52 2022] DELETE /identity/v3/projects/2d87977f834942e588078bcd16f21a2e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:52.441812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f0b42-3432-4098-88bc-b33c48f7cb50 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.442787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f0b42-3432-4098-88bc-b33c48f7cb50 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.442787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f0b42-3432-4098-88bc-b33c48f7cb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.462010 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-a07f0b42-3432-4098-88bc-b33c48f7cb50 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:52.463262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a07f0b42-3432-4098-88bc-b33c48f7cb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:52.488779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 975/1947] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:52.498228 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda2e6d-2b48-480f-987a-3eb966df437c None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.498601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda2e6d-2b48-480f-987a-3eb966df437c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.498963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eda2e6d-2b48-480f-987a-3eb966df437c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.517636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-3eda2e6d-2b48-480f-987a-3eb966df437c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:52.518855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3eda2e6d-2b48-480f-987a-3eb966df437c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:52.526932 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 976/1948] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:52:52.531300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda2e6d-2b48-480f-987a-3eb966df437c None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:52.533454 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 973/1949] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:52:52.540826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eda2e6d-2b48-480f-987a-3eb966df437c None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:52.541819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d034c-00c0-4f70-bc89-4159a40e08de None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.542053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d034c-00c0-4f70-bc89-4159a40e08de None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.542339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d034c-00c0-4f70-bc89-4159a40e08de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.545648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8cf632-39b7-4337-8b00-189f9f7e27e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:52.546909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cf632-39b7-4337-8b00-189f9f7e27e3 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.547194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cf632-39b7-4337-8b00-189f9f7e27e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.547476 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8cf632-39b7-4337-8b00-189f9f7e27e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.548257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8cf632-39b7-4337-8b00-189f9f7e27e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:52:52.548572 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8cf632-39b7-4337-8b00-189f9f7e27e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:52.550168 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8cf632-39b7-4337-8b00-189f9f7e27e3 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:52.561616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-ce2d034c-00c0-4f70-bc89-4159a40e08de None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:52.562789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 977/1950] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:52:52.562909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ce2d034c-00c0-4f70-bc89-4159a40e08de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:52.567255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8cf632-39b7-4337-8b00-189f9f7e27e3 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:52.571564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 974/1951] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:52:52.576629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d034c-00c0-4f70-bc89-4159a40e08de None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:52.578176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8cf632-39b7-4337-8b00-189f9f7e27e3 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:52.582858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956d9b3-6a15-49c8-ab55-1a402a65d93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:52.583530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956d9b3-6a15-49c8-ab55-1a402a65d93e admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.583821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956d9b3-6a15-49c8-ab55-1a402a65d93e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.584137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956d9b3-6a15-49c8-ab55-1a402a65d93e admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.584832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956d9b3-6a15-49c8-ab55-1a402a65d93e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:52.585778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d034c-00c0-4f70-bc89-4159a40e08de None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:52.586772 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956d9b3-6a15-49c8-ab55-1a402a65d93e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:52.590414 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc11732-eb49-4a93-bac7-114e1fa46d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:52.591073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc11732-eb49-4a93-bac7-114e1fa46d02 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.591352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc11732-eb49-4a93-bac7-114e1fa46d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.591625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc11732-eb49-4a93-bac7-114e1fa46d02 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.592384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc11732-eb49-4a93-bac7-114e1fa46d02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:52:52.592693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc11732-eb49-4a93-bac7-114e1fa46d02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:52.594283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc11732-eb49-4a93-bac7-114e1fa46d02 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:52.607672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 975/1952] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:52:52.612201 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc11732-eb49-4a93-bac7-114e1fa46d02 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:52.616007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 978/1953] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:52 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:52.620923 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956d9b3-6a15-49c8-ab55-1a402a65d93e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:52.622810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc11732-eb49-4a93-bac7-114e1fa46d02 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:52.694065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956d9b3-6a15-49c8-ab55-1a402a65d93e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:52.700196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5801a-6060-41c0-a289-0f9e690f7d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:52.701139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:52.701633 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5801a-6060-41c0-a289-0f9e690f7d4a admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.701633 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5801a-6060-41c0-a289-0f9e690f7d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.702209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5801a-6060-41c0-a289-0f9e690f7d4a admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.703260 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5801a-6060-41c0-a289-0f9e690f7d4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:52.705646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5801a-6060-41c0-a289-0f9e690f7d4a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:52.715223 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84946b18-248d-41ff-ab46-623e4e1892df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:52.716397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84946b18-248d-41ff-ab46-623e4e1892df admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.716838 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84946b18-248d-41ff-ab46-623e4e1892df admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.717474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84946b18-248d-41ff-ab46-623e4e1892df admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.718395 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84946b18-248d-41ff-ab46-623e4e1892df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:52.720825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84946b18-248d-41ff-ab46-623e4e1892df admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:52.736048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 979/1954] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:52 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:52.737639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5801a-6060-41c0-a289-0f9e690f7d4a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:52.751023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5801a-6060-41c0-a289-0f9e690f7d4a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:52.754380 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 976/1955] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:52 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:52:52.759040 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84946b18-248d-41ff-ab46-623e4e1892df admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:52.762436 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ecd351-a5a9-4b8c-8954-6a46106cbe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:52.763149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecd351-a5a9-4b8c-8954-6a46106cbe36 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.763480 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecd351-a5a9-4b8c-8954-6a46106cbe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.763767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecd351-a5a9-4b8c-8954-6a46106cbe36 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.764652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ecd351-a5a9-4b8c-8954-6a46106cbe36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:52.766645 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ecd351-a5a9-4b8c-8954-6a46106cbe36 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:52.791513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 980/1956] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:52.794916 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ecd351-a5a9-4b8c-8954-6a46106cbe36 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:52.826979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ecd351-a5a9-4b8c-8954-6a46106cbe36 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:52.847372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84946b18-248d-41ff-ab46-623e4e1892df admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:52.862967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5763d-4f60-401e-a17b-3323b7959c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:52.862967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5763d-4f60-401e-a17b-3323b7959c9f admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.862967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5763d-4f60-401e-a17b-3323b7959c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.862967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd5763d-4f60-401e-a17b-3323b7959c9f admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.866077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5763d-4f60-401e-a17b-3323b7959c9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:52.868061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd5763d-4f60-401e-a17b-3323b7959c9f admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:52.894957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:52.896795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 977/1957] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:52 2022] POST /identity/v3/users => generated 288 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:52.900919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5763d-4f60-401e-a17b-3323b7959c9f admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:52.916919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd5763d-4f60-401e-a17b-3323b7959c9f admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:52.922693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b1ba7-20f7-4f0d-8cbc-10787ee06df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:52.923428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b1ba7-20f7-4f0d-8cbc-10787ee06df2 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.923710 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b1ba7-20f7-4f0d-8cbc-10787ee06df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.924014 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b1ba7-20f7-4f0d-8cbc-10787ee06df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.924566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe70dd9-6521-4243-bfe4-28264620c5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:52.924810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b1ba7-20f7-4f0d-8cbc-10787ee06df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:52.925548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe70dd9-6521-4243-bfe4-28264620c5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.925925 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe70dd9-6521-4243-bfe4-28264620c5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.926279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe70dd9-6521-4243-bfe4-28264620c5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.926726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b1ba7-20f7-4f0d-8cbc-10787ee06df2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:52.927259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe70dd9-6521-4243-bfe4-28264620c5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:52.929092 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe70dd9-6521-4243-bfe4-28264620c5c0 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:52.942546 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 978/1958] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:52.942943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 981/1959] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:52:52.943911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b1ba7-20f7-4f0d-8cbc-10787ee06df2 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:52.948158 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe70dd9-6521-4243-bfe4-28264620c5c0 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:52.956677 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b1ba7-20f7-4f0d-8cbc-10787ee06df2 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:52.962108 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5281e49-80cf-4d77-ace0-d39b77fb1441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:52.962862 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5281e49-80cf-4d77-ace0-d39b77fb1441 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.963153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5281e49-80cf-4d77-ace0-d39b77fb1441 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.963803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5281e49-80cf-4d77-ace0-d39b77fb1441 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.964615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5281e49-80cf-4d77-ace0-d39b77fb1441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:52.966210 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5281e49-80cf-4d77-ace0-d39b77fb1441 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:52.968215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe70dd9-6521-4243-bfe4-28264620c5c0 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:52.975186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87924cbd-eb05-43ae-ad66-7fc02a47b9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:52.976646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87924cbd-eb05-43ae-ad66-7fc02a47b9b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:52.977007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87924cbd-eb05-43ae-ad66-7fc02a47b9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:52.977362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87924cbd-eb05-43ae-ad66-7fc02a47b9b6 admin admin] PATH_INFO: `/v3/projects/85286248f2674f6cb87d204d2abb0ccf/users/5d13bc1932d948928ccd65e2b88700ca/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:52.980826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 979/1960] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:52:52.985206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5281e49-80cf-4d77-ace0-d39b77fb1441 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:53.002133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87924cbd-eb05-43ae-ad66-7fc02a47b9b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85286248f2674f6cb87d204d2abb0ccf, user_id=5d13bc1932d948928ccd65e2b88700ca, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:53.006071 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87924cbd-eb05-43ae-ad66-7fc02a47b9b6 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:53.011202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5281e49-80cf-4d77-ace0-d39b77fb1441 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:53.016879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89972a0d-eaf2-4811-8286-c2bbdcfb824f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:53.018022 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89972a0d-eaf2-4811-8286-c2bbdcfb824f admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.018374 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89972a0d-eaf2-4811-8286-c2bbdcfb824f admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.018753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89972a0d-eaf2-4811-8286-c2bbdcfb824f admin admin] PATH_INFO: `/v3/projects/229f45739b554b77976c7c785c9f961b/users/c5848f37651c44feb41f6c450a48c30c/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.022504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 982/1961] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:52 2022] PUT /identity/v3/projects/85286248f2674f6cb87d204d2abb0ccf/users/5d13bc1932d948928ccd65e2b88700ca/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:53.036858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89972a0d-eaf2-4811-8286-c2bbdcfb824f admin admin] RBAC: Authorizing `identity:create_grant(project_id=229f45739b554b77976c7c785c9f961b, user_id=c5848f37651c44feb41f6c450a48c30c, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:53.037827 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992afda-3a10-4a4e-aed9-6905436fc7e4 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.038125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992afda-3a10-4a4e-aed9-6905436fc7e4 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.038400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992afda-3a10-4a4e-aed9-6905436fc7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.039821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89972a0d-eaf2-4811-8286-c2bbdcfb824f admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:53.063176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 980/1962] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:52 2022] PUT /identity/v3/projects/229f45739b554b77976c7c785c9f961b/users/c5848f37651c44feb41f6c450a48c30c/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:52:53.083597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b9f16-8ff9-4b7b-b22f-11767492d988 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.083916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b9f16-8ff9-4b7b-b22f-11767492d988 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.084292 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083b9f16-8ff9-4b7b-b22f-11767492d988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.094655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-1992afda-3a10-4a4e-aed9-6905436fc7e4 None None] MFA Rules not processed for user `5d13bc1932d948928ccd65e2b88700ca`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:53.096443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1992afda-3a10-4a4e-aed9-6905436fc7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:53.138838 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-083b9f16-8ff9-4b7b-b22f-11767492d988 None None] MFA Rules not processed for user `c5848f37651c44feb41f6c450a48c30c`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:53.140587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-083b9f16-8ff9-4b7b-b22f-11767492d988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:53.218741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 983/1963] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:53.222132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992afda-3a10-4a4e-aed9-6905436fc7e4 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:53.240318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 981/1964] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:53.246470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b9f16-8ff9-4b7b-b22f-11767492d988 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:53.261367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:53.311905 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992afda-3a10-4a4e-aed9-6905436fc7e4 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:53.317584 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270452f-56fb-4c3b-919d-4e3d55ab126f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:53.319093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270452f-56fb-4c3b-919d-4e3d55ab126f admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.319471 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270452f-56fb-4c3b-919d-4e3d55ab126f admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.319803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270452f-56fb-4c3b-919d-4e3d55ab126f admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.320638 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270452f-56fb-4c3b-919d-4e3d55ab126f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:53.324035 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270452f-56fb-4c3b-919d-4e3d55ab126f admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:53.349960 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083b9f16-8ff9-4b7b-b22f-11767492d988 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:53.354639 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e9b2a-f83a-416d-98ba-2b4649ed96c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:53.355333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e9b2a-f83a-416d-98ba-2b4649ed96c8 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.355593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e9b2a-f83a-416d-98ba-2b4649ed96c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.355870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e9b2a-f83a-416d-98ba-2b4649ed96c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.356610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e9b2a-f83a-416d-98ba-2b4649ed96c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:53.358518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e9b2a-f83a-416d-98ba-2b4649ed96c8 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:53.360857 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 984/1965] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:53 2022] POST /identity/v3/projects => generated 387 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:53.365042 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270452f-56fb-4c3b-919d-4e3d55ab126f admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:53.392882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 982/1966] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:53 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:52:53.397316 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e9b2a-f83a-416d-98ba-2b4649ed96c8 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:53.407306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:53.441714 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270452f-56fb-4c3b-919d-4e3d55ab126f admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:53.514545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ed7da-96f5-45f8-a3bc-0a93654f8c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:53.515522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ed7da-96f5-45f8-a3bc-0a93654f8c02 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.515807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ed7da-96f5-45f8-a3bc-0a93654f8c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.516110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ed7da-96f5-45f8-a3bc-0a93654f8c02 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.516804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ed7da-96f5-45f8-a3bc-0a93654f8c02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:53.518664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e9b2a-f83a-416d-98ba-2b4649ed96c8 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:53.518784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ed7da-96f5-45f8-a3bc-0a93654f8c02 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:53.527131 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bd93a-557e-44cf-99e2-d02bb2381109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:53.528285 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bd93a-557e-44cf-99e2-d02bb2381109 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.529000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bd93a-557e-44cf-99e2-d02bb2381109 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.529503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3bd93a-557e-44cf-99e2-d02bb2381109 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.530823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3bd93a-557e-44cf-99e2-d02bb2381109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:53.535093 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3bd93a-557e-44cf-99e2-d02bb2381109 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:53.540866 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 985/1967] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:53 2022] POST /identity/v3/users => generated 300 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:53.545191 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ed7da-96f5-45f8-a3bc-0a93654f8c02 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:53.555699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ed7da-96f5-45f8-a3bc-0a93654f8c02 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:53.561089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ac558-0811-40ec-b8a3-debabb61f2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:53.561775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac558-0811-40ec-b8a3-debabb61f2fc admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.562050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac558-0811-40ec-b8a3-debabb61f2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.562479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ac558-0811-40ec-b8a3-debabb61f2fc admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.563360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ac558-0811-40ec-b8a3-debabb61f2fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:53.565256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ac558-0811-40ec-b8a3-debabb61f2fc admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:53.567778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 983/1968] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:53 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:53.571279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bd93a-557e-44cf-99e2-d02bb2381109 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:53.581789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3bd93a-557e-44cf-99e2-d02bb2381109 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:53.583909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 986/1969] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:53.587121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89613d-fcc9-45a1-9621-5eaed5c93590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:53.588647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ac558-0811-40ec-b8a3-debabb61f2fc admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:53.588825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89613d-fcc9-45a1-9621-5eaed5c93590 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.589313 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89613d-fcc9-45a1-9621-5eaed5c93590 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.589806 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89613d-fcc9-45a1-9621-5eaed5c93590 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.591140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89613d-fcc9-45a1-9621-5eaed5c93590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:53.594137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a89613d-fcc9-45a1-9621-5eaed5c93590 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:53.598454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ac558-0811-40ec-b8a3-debabb61f2fc admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:53.603133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e19a4c-3d0d-41c7-8abe-a78eadabf1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:53.604186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e19a4c-3d0d-41c7-8abe-a78eadabf1ea admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.604466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e19a4c-3d0d-41c7-8abe-a78eadabf1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.604745 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e19a4c-3d0d-41c7-8abe-a78eadabf1ea admin admin] PATH_INFO: `/v3/projects/9a8fa38584004e6598557db10bfa1f67/users/0454981ee3ff4ff588e2af8bd540abe2/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.611401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 984/1970] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:52:53.612462 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89613d-fcc9-45a1-9621-5eaed5c93590 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:53.620568 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e19a4c-3d0d-41c7-8abe-a78eadabf1ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a8fa38584004e6598557db10bfa1f67, user_id=0454981ee3ff4ff588e2af8bd540abe2, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:53.623229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e19a4c-3d0d-41c7-8abe-a78eadabf1ea admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:53.623343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89613d-fcc9-45a1-9621-5eaed5c93590 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:53.628071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8f2b9-7f30-4e94-8b3d-a402fa9e7ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:53.629079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8f2b9-7f30-4e94-8b3d-a402fa9e7ded admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.629512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8f2b9-7f30-4e94-8b3d-a402fa9e7ded admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.629793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8f2b9-7f30-4e94-8b3d-a402fa9e7ded admin admin] PATH_INFO: `/v3/projects/c7ff38b3f5fb4c2181c463025bdb9ead/users/c110d4a9ffab463d95ed7e6c41e17682/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.637458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 987/1971] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:53 2022] PUT /identity/v3/projects/9a8fa38584004e6598557db10bfa1f67/users/0454981ee3ff4ff588e2af8bd540abe2/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:52:53.641403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e19a4c-3d0d-41c7-8abe-a78eadabf1ea admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:53.647418 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8f2b9-7f30-4e94-8b3d-a402fa9e7ded admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ff38b3f5fb4c2181c463025bdb9ead, user_id=c110d4a9ffab463d95ed7e6c41e17682, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:53.650052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8f2b9-7f30-4e94-8b3d-a402fa9e7ded admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:53.665305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 985/1972] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:53 2022] PUT /identity/v3/projects/c7ff38b3f5fb4c2181c463025bdb9ead/users/c110d4a9ffab463d95ed7e6c41e17682/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:52:53.669162 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8f2b9-7f30-4e94-8b3d-a402fa9e7ded admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:53.706037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e19a4c-3d0d-41c7-8abe-a78eadabf1ea admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:53.721025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:53.741675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8f2b9-7f30-4e94-8b3d-a402fa9e7ded admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:53.747543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d67e07-e5c6-42d0-8871-f528d7df507b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:53.748386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d67e07-e5c6-42d0-8871-f528d7df507b admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.748678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d67e07-e5c6-42d0-8871-f528d7df507b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.748855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:53.749002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d67e07-e5c6-42d0-8871-f528d7df507b admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.749725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d67e07-e5c6-42d0-8871-f528d7df507b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:53.751512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d67e07-e5c6-42d0-8871-f528d7df507b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:53.767551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 986/1973] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:52:53.768671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d67e07-e5c6-42d0-8871-f528d7df507b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:53.775027 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a16d6-cb37-4950-8f7d-6fe21e75fe3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:53.776483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a16d6-cb37-4950-8f7d-6fe21e75fe3e admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.776774 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a16d6-cb37-4950-8f7d-6fe21e75fe3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.777064 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a16d6-cb37-4950-8f7d-6fe21e75fe3e admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.777839 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a16d6-cb37-4950-8f7d-6fe21e75fe3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:53.779455 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d67e07-e5c6-42d0-8871-f528d7df507b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:53.779573 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a16d6-cb37-4950-8f7d-6fe21e75fe3e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:53.784347 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562d292-215c-4a59-80d9-e37dfc6c7bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:53.785247 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562d292-215c-4a59-80d9-e37dfc6c7bbd admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.785540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562d292-215c-4a59-80d9-e37dfc6c7bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.785791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562d292-215c-4a59-80d9-e37dfc6c7bbd admin admin] PATH_INFO: `/v3/projects/c7ff38b3f5fb4c2181c463025bdb9ead/users/c110d4a9ffab463d95ed7e6c41e17682/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.787805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562d292-215c-4a59-80d9-e37dfc6c7bbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ff38b3f5fb4c2181c463025bdb9ead, user_id=c110d4a9ffab463d95ed7e6c41e17682, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:53.790455 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562d292-215c-4a59-80d9-e37dfc6c7bbd admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:53.793827 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 988/1974] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:53 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:52:53.794698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a16d6-cb37-4950-8f7d-6fe21e75fe3e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:53.801850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 987/1975] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:53 2022] PUT /identity/v3/projects/c7ff38b3f5fb4c2181c463025bdb9ead/users/c110d4a9ffab463d95ed7e6c41e17682/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:52:53.807465 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a16d6-cb37-4950-8f7d-6fe21e75fe3e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:53.812127 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36c0c-d405-41d3-af7a-d38c242de09c None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.812423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36c0c-d405-41d3-af7a-d38c242de09c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.812706 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36c0c-d405-41d3-af7a-d38c242de09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.842072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-ddf36c0c-d405-41d3-af7a-d38c242de09c None None] MFA Rules not processed for user `c110d4a9ffab463d95ed7e6c41e17682`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:53.843294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ddf36c0c-d405-41d3-af7a-d38c242de09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:53.867348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefc2db4-13ea-4749-948f-1efa8b5a1a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:53.868537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc2db4-13ea-4749-948f-1efa8b5a1a0e admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.868825 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc2db4-13ea-4749-948f-1efa8b5a1a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.869130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefc2db4-13ea-4749-948f-1efa8b5a1a0e admin admin] PATH_INFO: `/v3/projects/9a8fa38584004e6598557db10bfa1f67/users/0454981ee3ff4ff588e2af8bd540abe2/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.871198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc2db4-13ea-4749-948f-1efa8b5a1a0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a8fa38584004e6598557db10bfa1f67, user_id=0454981ee3ff4ff588e2af8bd540abe2, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:53.874045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefc2db4-13ea-4749-948f-1efa8b5a1a0e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:53.896740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 989/1976] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:53 2022] PUT /identity/v3/projects/9a8fa38584004e6598557db10bfa1f67/users/0454981ee3ff4ff588e2af8bd540abe2/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:52:53.908567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba58c6-9f54-4e62-9deb-3d5e34ff1d87 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:53.908938 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba58c6-9f54-4e62-9deb-3d5e34ff1d87 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:53.909439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ba58c6-9f54-4e62-9deb-3d5e34ff1d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:53.926185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 988/1977] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:52:53.938019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf36c0c-d405-41d3-af7a-d38c242de09c None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:53.939304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b9ba58c6-9f54-4e62-9deb-3d5e34ff1d87 None None] MFA Rules not processed for user `0454981ee3ff4ff588e2af8bd540abe2`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:53.939601 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ddf36c0c-d405-41d3-af7a-d38c242de09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:53.940830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b9ba58c6-9f54-4e62-9deb-3d5e34ff1d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:54.009147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf36c0c-d405-41d3-af7a-d38c242de09c None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:54.028506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 990/1978] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:52:54.040234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba58c6-9f54-4e62-9deb-3d5e34ff1d87 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:54.041729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b9ba58c6-9f54-4e62-9deb-3d5e34ff1d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:54.075220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7197c592-d7c0-4efc-9d33-eab9873c072c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:54.075884 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197c592-d7c0-4efc-9d33-eab9873c072c service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:54.076202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197c592-d7c0-4efc-9d33-eab9873c072c service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:54.076480 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7197c592-d7c0-4efc-9d33-eab9873c072c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:54.110958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:54.111791 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ba58c6-9f54-4e62-9deb-3d5e34ff1d87 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:54.121964 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d08855-d3fe-4743-80d4-50531823719d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:54.122654 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d08855-d3fe-4743-80d4-50531823719d service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:54.122945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d08855-d3fe-4743-80d4-50531823719d service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:54.123225 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d08855-d3fe-4743-80d4-50531823719d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:54.133282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d08855-d3fe-4743-80d4-50531823719d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:54.135294 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d08855-d3fe-4743-80d4-50531823719d service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:54.140310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:54.149681 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 991/1979] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:52:54.154411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197c592-d7c0-4efc-9d33-eab9873c072c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:54.155863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7197c592-d7c0-4efc-9d33-eab9873c072c service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:54.167736 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 989/1980] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:52:54.178633 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d08855-d3fe-4743-80d4-50531823719d service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:54.255701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d08855-d3fe-4743-80d4-50531823719d service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:54.261000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6996c3-76f1-47af-a074-85c9f8535848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:54.261978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6996c3-76f1-47af-a074-85c9f8535848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:54.262340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6996c3-76f1-47af-a074-85c9f8535848 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:54.262710 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6996c3-76f1-47af-a074-85c9f8535848 admin admin] PATH_INFO: `/v3/users/5d13bc1932d948928ccd65e2b88700ca` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:54.264253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6996c3-76f1-47af-a074-85c9f8535848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d13bc1932d948928ccd65e2b88700ca)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:54.266246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6996c3-76f1-47af-a074-85c9f8535848 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:54.320629 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1a6996c3-76f1-47af-a074-85c9f8535848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d13bc1932d948928ccd65e2b88700ca'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:54.336226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1a6996c3-76f1-47af-a074-85c9f8535848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d13bc1932d948928ccd65e2b88700ca'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:54.344085 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1a6996c3-76f1-47af-a074-85c9f8535848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d13bc1932d948928ccd65e2b88700ca'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:54.344925 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1a6996c3-76f1-47af-a074-85c9f8535848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d13bc1932d948928ccd65e2b88700ca'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:54.352608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1a6996c3-76f1-47af-a074-85c9f8535848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d13bc1932d948928ccd65e2b88700ca'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:54.359955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 992/1981] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:54 2022] DELETE /identity/v3/users/5d13bc1932d948928ccd65e2b88700ca => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:54.364960 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a85b5e-2efb-4d96-b1b2-1123bfcb9791 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:54.365250 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a85b5e-2efb-4d96-b1b2-1123bfcb9791 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:54.365528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a85b5e-2efb-4d96-b1b2-1123bfcb9791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:54.384204 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-45a85b5e-2efb-4d96-b1b2-1123bfcb9791 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:54.385504 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-45a85b5e-2efb-4d96-b1b2-1123bfcb9791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:54.448675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 990/1982] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:52:54.459620 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6996c3-76f1-47af-a074-85c9f8535848 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:54.461142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1a6996c3-76f1-47af-a074-85c9f8535848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:54.547449 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6996c3-76f1-47af-a074-85c9f8535848 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:54.562117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8ed2d-8e18-4a87-936e-e4db9217b901 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:54.563407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8ed2d-8e18-4a87-936e-e4db9217b901 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:54.563407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8ed2d-8e18-4a87-936e-e4db9217b901 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:54.564192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead8ed2d-8e18-4a87-936e-e4db9217b901 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:54.597367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8ed2d-8e18-4a87-936e-e4db9217b901 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:54.599000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead8ed2d-8e18-4a87-936e-e4db9217b901 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:54.614268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 993/1983] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:55.017015 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a85b5e-2efb-4d96-b1b2-1123bfcb9791 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:55.018680 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-45a85b5e-2efb-4d96-b1b2-1123bfcb9791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:55.032068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a85b5e-2efb-4d96-b1b2-1123bfcb9791 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:55.036618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:55.037486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:55.037766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:55.038050 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] PATH_INFO: `/v3/projects/85286248f2674f6cb87d204d2abb0ccf` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:55.039268 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85286248f2674f6cb87d204d2abb0ccf)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:55.041775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:55.098808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85286248f2674f6cb87d204d2abb0ccf', 'request_id': 'req-382f3adc-52b6-4121-a3c9-0ce145f1ed16'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.107153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85286248f2674f6cb87d204d2abb0ccf', 'request_id': 'req-382f3adc-52b6-4121-a3c9-0ce145f1ed16'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.115738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85286248f2674f6cb87d204d2abb0ccf', 'request_id': 'req-382f3adc-52b6-4121-a3c9-0ce145f1ed16'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.135589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] The token cache is being invalidate because project 85286248f2674f6cb87d204d2abb0ccf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:55.136131 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.137218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.139816 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 991/1984] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:55 2022] DELETE /identity/v3/projects/85286248f2674f6cb87d204d2abb0ccf => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:55.144385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8ed2d-8e18-4a87-936e-e4db9217b901 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:55.155495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead8ed2d-8e18-4a87-936e-e4db9217b901 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:55.172971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:55.231128 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932fc757-4e50-41bb-9af9-ba1797531027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:55.231128 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fc757-4e50-41bb-9af9-ba1797531027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:55.231128 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fc757-4e50-41bb-9af9-ba1797531027 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:55.231492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932fc757-4e50-41bb-9af9-ba1797531027 admin admin] PATH_INFO: `/v3/users/0454981ee3ff4ff588e2af8bd540abe2` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:55.232629 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fc757-4e50-41bb-9af9-ba1797531027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0454981ee3ff4ff588e2af8bd540abe2)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:55.234554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932fc757-4e50-41bb-9af9-ba1797531027 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:55.291446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-932fc757-4e50-41bb-9af9-ba1797531027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0454981ee3ff4ff588e2af8bd540abe2'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.307980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-932fc757-4e50-41bb-9af9-ba1797531027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0454981ee3ff4ff588e2af8bd540abe2'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.323644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-932fc757-4e50-41bb-9af9-ba1797531027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0454981ee3ff4ff588e2af8bd540abe2'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.324330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-932fc757-4e50-41bb-9af9-ba1797531027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0454981ee3ff4ff588e2af8bd540abe2'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.331829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-932fc757-4e50-41bb-9af9-ba1797531027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0454981ee3ff4ff588e2af8bd540abe2'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.344146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 994/1985] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:55 2022] DELETE /identity/v3/users/0454981ee3ff4ff588e2af8bd540abe2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:55.732573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:55.734239 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:55.747872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382f3adc-52b6-4121-a3c9-0ce145f1ed16 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:55.805476 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:55.819543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265a7134-6164-4a5d-8021-9944b70c4be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:55.820461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a7134-6164-4a5d-8021-9944b70c4be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:55.820739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a7134-6164-4a5d-8021-9944b70c4be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:55.821012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a7134-6164-4a5d-8021-9944b70c4be7 admin admin] PATH_INFO: `/v3/projects/9a8fa38584004e6598557db10bfa1f67` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:55.822176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a7134-6164-4a5d-8021-9944b70c4be7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a8fa38584004e6598557db10bfa1f67)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:55.824633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265a7134-6164-4a5d-8021-9944b70c4be7 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:55.871418 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-265a7134-6164-4a5d-8021-9944b70c4be7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a8fa38584004e6598557db10bfa1f67', 'request_id': 'req-265a7134-6164-4a5d-8021-9944b70c4be7'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.879858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-265a7134-6164-4a5d-8021-9944b70c4be7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a8fa38584004e6598557db10bfa1f67', 'request_id': 'req-265a7134-6164-4a5d-8021-9944b70c4be7'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.886093 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-265a7134-6164-4a5d-8021-9944b70c4be7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a8fa38584004e6598557db10bfa1f67', 'request_id': 'req-265a7134-6164-4a5d-8021-9944b70c4be7'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.897074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-265a7134-6164-4a5d-8021-9944b70c4be7 admin admin] The token cache is being invalidate because project 9a8fa38584004e6598557db10bfa1f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:55.897360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-265a7134-6164-4a5d-8021-9944b70c4be7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.898033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-265a7134-6164-4a5d-8021-9944b70c4be7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:55.899584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 992/1986] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:55 2022] DELETE /identity/v3/projects/9a8fa38584004e6598557db10bfa1f67 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:55.905165 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db55e22-4015-4616-9165-2f6d2a4e240b None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:55.905462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db55e22-4015-4616-9165-2f6d2a4e240b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:55.905741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db55e22-4015-4616-9165-2f6d2a4e240b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:55.924356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-1db55e22-4015-4616-9165-2f6d2a4e240b None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:55.925568 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1db55e22-4015-4616-9165-2f6d2a4e240b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:55.992208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 995/1987] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:56.000766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee5106-938b-4302-a4bf-436f191c6249 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:56.001083 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee5106-938b-4302-a4bf-436f191c6249 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:56.001695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee5106-938b-4302-a4bf-436f191c6249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:56.020953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-8bee5106-938b-4302-a4bf-436f191c6249 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:56.022213 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-8bee5106-938b-4302-a4bf-436f191c6249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:56.031235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 993/1988] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:52:56.035461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db55e22-4015-4616-9165-2f6d2a4e240b None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:56.049067 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db55e22-4015-4616-9165-2f6d2a4e240b None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:56.054259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057422d-932e-44dd-9903-78449328a82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:56.054938 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057422d-932e-44dd-9903-78449328a82b admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:56.055420 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057422d-932e-44dd-9903-78449328a82b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:56.055538 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b057422d-932e-44dd-9903-78449328a82b admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:56.056279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057422d-932e-44dd-9903-78449328a82b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:52:56.056591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057422d-932e-44dd-9903-78449328a82b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:56.058771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b057422d-932e-44dd-9903-78449328a82b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:56.071540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 996/1989] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:52:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:52:56.075896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee5106-938b-4302-a4bf-436f191c6249 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:56.085132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee5106-938b-4302-a4bf-436f191c6249 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:56.090004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4be5a-5076-45da-9963-1ac2cc9d7fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:56.090722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4be5a-5076-45da-9963-1ac2cc9d7fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:56.090967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4be5a-5076-45da-9963-1ac2cc9d7fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:56.091240 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4be5a-5076-45da-9963-1ac2cc9d7fa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:56.091976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4be5a-5076-45da-9963-1ac2cc9d7fa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:56.099411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4be5a-5076-45da-9963-1ac2cc9d7fa9 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:56.149086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 994/1990] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:56 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:56.153616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057422d-932e-44dd-9903-78449328a82b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:56.163876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b057422d-932e-44dd-9903-78449328a82b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:56.188426 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:56.222001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc9aa3-a4ac-4ef3-a673-2b5be315c877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:56.222679 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc9aa3-a4ac-4ef3-a673-2b5be315c877 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:56.222967 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc9aa3-a4ac-4ef3-a673-2b5be315c877 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:56.223250 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc9aa3-a4ac-4ef3-a673-2b5be315c877 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:56.223960 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc9aa3-a4ac-4ef3-a673-2b5be315c877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:56.225878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfc9aa3-a4ac-4ef3-a673-2b5be315c877 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:56.248468 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 997/1991] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:56 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:56.252443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4be5a-5076-45da-9963-1ac2cc9d7fa9 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:56.267095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4be5a-5076-45da-9963-1ac2cc9d7fa9 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:56.276607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c840e9-51c7-4686-9ae9-b3a548785e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:56.277301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c840e9-51c7-4686-9ae9-b3a548785e36 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:56.277593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c840e9-51c7-4686-9ae9-b3a548785e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:56.277872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c840e9-51c7-4686-9ae9-b3a548785e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:56.278613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c840e9-51c7-4686-9ae9-b3a548785e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:56.280301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c840e9-51c7-4686-9ae9-b3a548785e36 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:56.293902 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 995/1992] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:56.297812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc9aa3-a4ac-4ef3-a673-2b5be315c877 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:56.308356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfc9aa3-a4ac-4ef3-a673-2b5be315c877 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:56.313167 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fde826-e7db-4b4e-adcf-fedd696584f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:56.314162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fde826-e7db-4b4e-adcf-fedd696584f5 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:56.314162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fde826-e7db-4b4e-adcf-fedd696584f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:56.314370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fde826-e7db-4b4e-adcf-fedd696584f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:56.315386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fde826-e7db-4b4e-adcf-fedd696584f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:56.317125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fde826-e7db-4b4e-adcf-fedd696584f5 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:56.336473 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 998/1993] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:52:56.344854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c840e9-51c7-4686-9ae9-b3a548785e36 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:56.357369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c840e9-51c7-4686-9ae9-b3a548785e36 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:56.364378 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae4290-94cd-430e-91c3-c46ed873ead8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:56.365290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae4290-94cd-430e-91c3-c46ed873ead8 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:56.365570 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae4290-94cd-430e-91c3-c46ed873ead8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:56.365874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ae4290-94cd-430e-91c3-c46ed873ead8 admin admin] PATH_INFO: `/v3/projects/7f9bab2f74834f07b75d0b697068bfa8/users/a825f40528e146dbb885bbf023be459b/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:56.383477 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae4290-94cd-430e-91c3-c46ed873ead8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f9bab2f74834f07b75d0b697068bfa8, user_id=a825f40528e146dbb885bbf023be459b, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:56.388497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ae4290-94cd-430e-91c3-c46ed873ead8 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:56.418025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 996/1994] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:56 2022] PUT /identity/v3/projects/7f9bab2f74834f07b75d0b697068bfa8/users/a825f40528e146dbb885bbf023be459b/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:56.437841 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543ed69-8fad-426d-97e6-e3888b43e50b None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:56.438237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543ed69-8fad-426d-97e6-e3888b43e50b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:56.438600 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543ed69-8fad-426d-97e6-e3888b43e50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:56.492794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b543ed69-8fad-426d-97e6-e3888b43e50b None None] MFA Rules not processed for user `a825f40528e146dbb885bbf023be459b`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:56.495487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b543ed69-8fad-426d-97e6-e3888b43e50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:56.640962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 999/1995] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:56.655039 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae4290-94cd-430e-91c3-c46ed873ead8 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:56.758051 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ae4290-94cd-430e-91c3-c46ed873ead8 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:56.773287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3a695-66cc-400b-9588-0a8671e3ddfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:56.773287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3a695-66cc-400b-9588-0a8671e3ddfc admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:56.773287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3a695-66cc-400b-9588-0a8671e3ddfc admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:56.773287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc3a695-66cc-400b-9588-0a8671e3ddfc admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:56.773941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3a695-66cc-400b-9588-0a8671e3ddfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:56.774666 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc3a695-66cc-400b-9588-0a8671e3ddfc admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:56.816116 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:56.828177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 997/1996] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:56 2022] POST /identity/v3/projects => generated 363 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:56.828642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543ed69-8fad-426d-97e6-e3888b43e50b None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:56.845761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543ed69-8fad-426d-97e6-e3888b43e50b None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:56.990620 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfecf10-b2b9-40eb-ac23-2d31247dde29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:56.991688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfecf10-b2b9-40eb-ac23-2d31247dde29 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:56.992088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfecf10-b2b9-40eb-ac23-2d31247dde29 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:56.992624 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfecf10-b2b9-40eb-ac23-2d31247dde29 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:56.993491 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfecf10-b2b9-40eb-ac23-2d31247dde29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:56.998326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfecf10-b2b9-40eb-ac23-2d31247dde29 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:57.031156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1000/1997] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:56 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:52:57.034825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3a695-66cc-400b-9588-0a8671e3ddfc admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:57.046161 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc3a695-66cc-400b-9588-0a8671e3ddfc admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:57.051707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5c519-5db5-442f-a556-ebfa2da5e5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:57.052496 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5c519-5db5-442f-a556-ebfa2da5e5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:57.054067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5c519-5db5-442f-a556-ebfa2da5e5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:57.054067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa5c519-5db5-442f-a556-ebfa2da5e5b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:57.054067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa5c519-5db5-442f-a556-ebfa2da5e5b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:57.055577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa5c519-5db5-442f-a556-ebfa2da5e5b3 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:57.068920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 998/1998] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:57.073560 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfecf10-b2b9-40eb-ac23-2d31247dde29 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:57.089257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfecf10-b2b9-40eb-ac23-2d31247dde29 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:57.096212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3968b8-825f-4493-93b6-0dc6c0f88ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:57.098628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3968b8-825f-4493-93b6-0dc6c0f88ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:57.098628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3968b8-825f-4493-93b6-0dc6c0f88ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:57.098628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3968b8-825f-4493-93b6-0dc6c0f88ef1 admin admin] PATH_INFO: `/v3/projects/fedd4b712f394cd1b74ae72a51e91aa9/users/377bc5f882714749bf5870df1703e691/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:57.115280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3968b8-825f-4493-93b6-0dc6c0f88ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fedd4b712f394cd1b74ae72a51e91aa9, user_id=377bc5f882714749bf5870df1703e691, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:57.118460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3968b8-825f-4493-93b6-0dc6c0f88ef1 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:57.135485 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:57.141952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1001/1999] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:57 2022] PUT /identity/v3/projects/fedd4b712f394cd1b74ae72a51e91aa9/users/377bc5f882714749bf5870df1703e691/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:57.146315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5c519-5db5-442f-a556-ebfa2da5e5b3 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:57.161511 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa5c519-5db5-442f-a556-ebfa2da5e5b3 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:57.227917 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b036c664-a924-4ac1-b2a1-206d39ffeaa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:57.228663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036c664-a924-4ac1-b2a1-206d39ffeaa6 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:57.229443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036c664-a924-4ac1-b2a1-206d39ffeaa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:57.229765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b036c664-a924-4ac1-b2a1-206d39ffeaa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:57.230509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036c664-a924-4ac1-b2a1-206d39ffeaa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:57.232135 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b036c664-a924-4ac1-b2a1-206d39ffeaa6 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:57.243998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 999/2000] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:52:57 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:52:57.248490 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3968b8-825f-4493-93b6-0dc6c0f88ef1 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:57.259677 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3968b8-825f-4493-93b6-0dc6c0f88ef1 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:57.266152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094bbcf-8149-4b23-b5c4-a55d6660e228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:57.267306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094bbcf-8149-4b23-b5c4-a55d6660e228 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:57.267460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094bbcf-8149-4b23-b5c4-a55d6660e228 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:57.267641 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094bbcf-8149-4b23-b5c4-a55d6660e228 admin admin] PATH_INFO: `/v3/projects/fedd4b712f394cd1b74ae72a51e91aa9/users/377bc5f882714749bf5870df1703e691/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:57.269803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094bbcf-8149-4b23-b5c4-a55d6660e228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fedd4b712f394cd1b74ae72a51e91aa9, user_id=377bc5f882714749bf5870df1703e691, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:57.272537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094bbcf-8149-4b23-b5c4-a55d6660e228 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:57.284946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1002/2001] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:52:57 2022] PUT /identity/v3/projects/fedd4b712f394cd1b74ae72a51e91aa9/users/377bc5f882714749bf5870df1703e691/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:57.296276 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e443eb1-9eda-406e-a652-e2dbcd1c2727 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:57.296722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e443eb1-9eda-406e-a652-e2dbcd1c2727 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:57.297076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e443eb1-9eda-406e-a652-e2dbcd1c2727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:57.325673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-9e443eb1-9eda-406e-a652-e2dbcd1c2727 None None] MFA Rules not processed for user `377bc5f882714749bf5870df1703e691`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:57.327021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9e443eb1-9eda-406e-a652-e2dbcd1c2727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:57.407329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1000/2002] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:57 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:52:57.428496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094bbcf-8149-4b23-b5c4-a55d6660e228 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:57.433704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f094bbcf-8149-4b23-b5c4-a55d6660e228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:57.506775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094bbcf-8149-4b23-b5c4-a55d6660e228 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:57.510684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1fcfb-2830-4b9e-90b7-329de434189c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:57.511338 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1fcfb-2830-4b9e-90b7-329de434189c service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:57.511835 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1fcfb-2830-4b9e-90b7-329de434189c service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:57.511984 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1fcfb-2830-4b9e-90b7-329de434189c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:57.523137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1fcfb-2830-4b9e-90b7-329de434189c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:57.524627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1fcfb-2830-4b9e-90b7-329de434189c service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:57.535475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1003/2003] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:57 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:57.647248 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e443eb1-9eda-406e-a652-e2dbcd1c2727 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:57.664987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e443eb1-9eda-406e-a652-e2dbcd1c2727 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:57.677842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77240a-af94-4385-89eb-5c7c904d1482 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:57.678805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77240a-af94-4385-89eb-5c7c904d1482 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:57.679193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77240a-af94-4385-89eb-5c7c904d1482 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:57.679564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77240a-af94-4385-89eb-5c7c904d1482 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:57.827756 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77240a-af94-4385-89eb-5c7c904d1482 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:57.829269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77240a-af94-4385-89eb-5c7c904d1482 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:57.840916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1001/2004] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:58.192783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1fcfb-2830-4b9e-90b7-329de434189c service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:58.194616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-39c1fcfb-2830-4b9e-90b7-329de434189c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:58.279948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1fcfb-2830-4b9e-90b7-329de434189c service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:58.286402 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed133c-22f6-4a6d-a224-0be58a7fa119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:58.287684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed133c-22f6-4a6d-a224-0be58a7fa119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:58.288058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed133c-22f6-4a6d-a224-0be58a7fa119 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:58.288343 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed133c-22f6-4a6d-a224-0be58a7fa119 admin admin] PATH_INFO: `/v3/users/c5848f37651c44feb41f6c450a48c30c` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:58.289622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed133c-22f6-4a6d-a224-0be58a7fa119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5848f37651c44feb41f6c450a48c30c)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:58.292314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed133c-22f6-4a6d-a224-0be58a7fa119 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:58.347659 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-67ed133c-22f6-4a6d-a224-0be58a7fa119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5848f37651c44feb41f6c450a48c30c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:58.362409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-67ed133c-22f6-4a6d-a224-0be58a7fa119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5848f37651c44feb41f6c450a48c30c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:58.370970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-67ed133c-22f6-4a6d-a224-0be58a7fa119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5848f37651c44feb41f6c450a48c30c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:58.371605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-67ed133c-22f6-4a6d-a224-0be58a7fa119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5848f37651c44feb41f6c450a48c30c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:58.379004 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-67ed133c-22f6-4a6d-a224-0be58a7fa119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5848f37651c44feb41f6c450a48c30c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:58.385223 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1004/2005] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:58 2022] DELETE /identity/v3/users/c5848f37651c44feb41f6c450a48c30c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:58.389839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2826e87-45cd-4564-8507-d3783305ef24 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:58.390329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2826e87-45cd-4564-8507-d3783305ef24 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:58.390435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2826e87-45cd-4564-8507-d3783305ef24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:58.407912 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-b2826e87-45cd-4564-8507-d3783305ef24 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:58.409166 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b2826e87-45cd-4564-8507-d3783305ef24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:58.413195 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:58.501539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1002/2006] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:58.511301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed133c-22f6-4a6d-a224-0be58a7fa119 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:58.513277 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-67ed133c-22f6-4a6d-a224-0be58a7fa119 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:58.524286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:58.575790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed133c-22f6-4a6d-a224-0be58a7fa119 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:58.579407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44840ab9-2e5c-442d-97c9-3bf9857a83b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:58.580109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44840ab9-2e5c-442d-97c9-3bf9857a83b7 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:58.580393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44840ab9-2e5c-442d-97c9-3bf9857a83b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:58.580659 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44840ab9-2e5c-442d-97c9-3bf9857a83b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:58.590701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44840ab9-2e5c-442d-97c9-3bf9857a83b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:58.592169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44840ab9-2e5c-442d-97c9-3bf9857a83b7 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:58.601913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1005/2007] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:52:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:52:58.890871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2826e87-45cd-4564-8507-d3783305ef24 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:58.892352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b2826e87-45cd-4564-8507-d3783305ef24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:58.905238 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2826e87-45cd-4564-8507-d3783305ef24 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:58.909896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:58.910769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:58.911048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:58.911325 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] PATH_INFO: `/v3/projects/229f45739b554b77976c7c785c9f961b` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:58.912539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=229f45739b554b77976c7c785c9f961b)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:58.914467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:58.962868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '229f45739b554b77976c7c785c9f961b', 'request_id': 'req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:58.973196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '229f45739b554b77976c7c785c9f961b', 'request_id': 'req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:58.981761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '229f45739b554b77976c7c785c9f961b', 'request_id': 'req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:58.991318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] The token cache is being invalidate because project 229f45739b554b77976c7c785c9f961b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:58.991732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:58.992505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:58.994153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1003/2008] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:58 2022] DELETE /identity/v3/projects/229f45739b554b77976c7c785c9f961b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:58.998333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44840ab9-2e5c-442d-97c9-3bf9857a83b7 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:59.011418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44840ab9-2e5c-442d-97c9-3bf9857a83b7 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:59.075736 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f0199-fe71-4a8f-845d-f09f8feebfbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:59.076656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f0199-fe71-4a8f-845d-f09f8feebfbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:59.076946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f0199-fe71-4a8f-845d-f09f8feebfbb admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:59.077225 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f0199-fe71-4a8f-845d-f09f8feebfbb admin admin] PATH_INFO: `/v3/users/c110d4a9ffab463d95ed7e6c41e17682` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:59.078462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f0199-fe71-4a8f-845d-f09f8feebfbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c110d4a9ffab463d95ed7e6c41e17682)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:59.080457 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f0199-fe71-4a8f-845d-f09f8feebfbb admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:59.132742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3e9f0199-fe71-4a8f-845d-f09f8feebfbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c110d4a9ffab463d95ed7e6c41e17682'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:59.148279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3e9f0199-fe71-4a8f-845d-f09f8feebfbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c110d4a9ffab463d95ed7e6c41e17682'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:59.157137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3e9f0199-fe71-4a8f-845d-f09f8feebfbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c110d4a9ffab463d95ed7e6c41e17682'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:59.157781 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3e9f0199-fe71-4a8f-845d-f09f8feebfbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c110d4a9ffab463d95ed7e6c41e17682'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:59.165829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3e9f0199-fe71-4a8f-845d-f09f8feebfbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c110d4a9ffab463d95ed7e6c41e17682'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:59.179747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1006/2009] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:52:58 2022] DELETE /identity/v3/users/c110d4a9ffab463d95ed7e6c41e17682 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:59.512478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:59.514587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:59.528682 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae44f3e-b64d-49e5-b7e7-fd7dbea25751 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:59.596123 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cf700-7940-452a-a82b-cdd4d7e78022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:59.597268 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cf700-7940-452a-a82b-cdd4d7e78022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:59.597794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cf700-7940-452a-a82b-cdd4d7e78022 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:59.598013 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cf700-7940-452a-a82b-cdd4d7e78022 admin admin] PATH_INFO: `/v3/projects/c7ff38b3f5fb4c2181c463025bdb9ead` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:59.599094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cf700-7940-452a-a82b-cdd4d7e78022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7ff38b3f5fb4c2181c463025bdb9ead)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:59.601100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cf700-7940-452a-a82b-cdd4d7e78022 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:59.651786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:59.665420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ab2cf700-7940-452a-a82b-cdd4d7e78022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7ff38b3f5fb4c2181c463025bdb9ead', 'request_id': 'req-ab2cf700-7940-452a-a82b-cdd4d7e78022'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:59.674262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ab2cf700-7940-452a-a82b-cdd4d7e78022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7ff38b3f5fb4c2181c463025bdb9ead', 'request_id': 'req-ab2cf700-7940-452a-a82b-cdd4d7e78022'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:59.682101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ab2cf700-7940-452a-a82b-cdd4d7e78022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7ff38b3f5fb4c2181c463025bdb9ead', 'request_id': 'req-ab2cf700-7940-452a-a82b-cdd4d7e78022'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:59.697327 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ab2cf700-7940-452a-a82b-cdd4d7e78022 admin admin] The token cache is being invalidate because project c7ff38b3f5fb4c2181c463025bdb9ead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:52:59.698218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ab2cf700-7940-452a-a82b-cdd4d7e78022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:59.699015 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ab2cf700-7940-452a-a82b-cdd4d7e78022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:52:59.701043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1004/2010] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:52:59 2022] DELETE /identity/v3/projects/c7ff38b3f5fb4c2181c463025bdb9ead => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:52:59.706897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fc9d5-5e09-4c46-9143-424cd52314f9 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:59.707366 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fc9d5-5e09-4c46-9143-424cd52314f9 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:59.707569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153fc9d5-5e09-4c46-9143-424cd52314f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:59.730501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-153fc9d5-5e09-4c46-9143-424cd52314f9 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:59.731737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-153fc9d5-5e09-4c46-9143-424cd52314f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:59.762597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:52:59.800789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1007/2011] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:59.810306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b89a4e-e8f2-4692-a4a8-359437eddfc1 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:59.810601 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b89a4e-e8f2-4692-a4a8-359437eddfc1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:59.810881 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b89a4e-e8f2-4692-a4a8-359437eddfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:59.831100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-46b89a4e-e8f2-4692-a4a8-359437eddfc1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:52:59.832492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-46b89a4e-e8f2-4692-a4a8-359437eddfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:52:59.842502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1005/2012] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:52:59.846637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fc9d5-5e09-4c46-9143-424cd52314f9 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:59.856611 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153fc9d5-5e09-4c46-9143-424cd52314f9 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:59.861645 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047a823-efab-419b-947f-e486c89b4ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:59.862403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047a823-efab-419b-947f-e486c89b4ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:59.862780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047a823-efab-419b-947f-e486c89b4ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:59.863129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047a823-efab-419b-947f-e486c89b4ba2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:59.863986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047a823-efab-419b-947f-e486c89b4ba2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:52:59.864379 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047a823-efab-419b-947f-e486c89b4ba2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:59.866067 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047a823-efab-419b-947f-e486c89b4ba2 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:59.878641 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1008/2013] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:52:59.883145 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b89a4e-e8f2-4692-a4a8-359437eddfc1 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:59.892937 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b89a4e-e8f2-4692-a4a8-359437eddfc1 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:52:59.897886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86993268-71d8-423e-a486-6619c3a460ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:52:59.898568 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86993268-71d8-423e-a486-6619c3a460ab admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:52:59.898855 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86993268-71d8-423e-a486-6619c3a460ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:52:59.899137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86993268-71d8-423e-a486-6619c3a460ab admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:52:59.900044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86993268-71d8-423e-a486-6619c3a460ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:52:59.902573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86993268-71d8-423e-a486-6619c3a460ab admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:52:59.937335 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1006/2014] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:52:59 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:52:59.941850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047a823-efab-419b-947f-e486c89b4ba2 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:52:59.952512 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047a823-efab-419b-947f-e486c89b4ba2 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:00.015460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c298fe-2d46-4c81-8e45-81c21f0ccc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:00.016218 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c298fe-2d46-4c81-8e45-81c21f0ccc9d admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:00.016510 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c298fe-2d46-4c81-8e45-81c21f0ccc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:00.016795 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c298fe-2d46-4c81-8e45-81c21f0ccc9d admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:00.017502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c298fe-2d46-4c81-8e45-81c21f0ccc9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:00.019428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c298fe-2d46-4c81-8e45-81c21f0ccc9d admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:00.044500 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1009/2015] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:52:59 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:00.045750 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86993268-71d8-423e-a486-6619c3a460ab admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:00.057471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86993268-71d8-423e-a486-6619c3a460ab admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:00.062249 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4648f8-29a9-4124-ade4-d7a283b8dd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:00.062923 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4648f8-29a9-4124-ade4-d7a283b8dd11 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:00.063204 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4648f8-29a9-4124-ade4-d7a283b8dd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:00.063482 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4648f8-29a9-4124-ade4-d7a283b8dd11 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:00.064235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4648f8-29a9-4124-ade4-d7a283b8dd11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:00.066126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4648f8-29a9-4124-ade4-d7a283b8dd11 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:00.076906 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1007/2016] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:53:00.081068 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c298fe-2d46-4c81-8e45-81c21f0ccc9d admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:00.091896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c298fe-2d46-4c81-8e45-81c21f0ccc9d admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:00.096852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a84d3-6f4c-4afb-b496-ca6e9e2e01cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:00.097533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a84d3-6f4c-4afb-b496-ca6e9e2e01cf admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:00.097821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a84d3-6f4c-4afb-b496-ca6e9e2e01cf admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:00.098104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a84d3-6f4c-4afb-b496-ca6e9e2e01cf admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:00.099005 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a84d3-6f4c-4afb-b496-ca6e9e2e01cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:00.100850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a84d3-6f4c-4afb-b496-ca6e9e2e01cf admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:00.116105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1010/2017] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:53:00.117564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4648f8-29a9-4124-ade4-d7a283b8dd11 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:00.127902 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4648f8-29a9-4124-ade4-d7a283b8dd11 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:00.132466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37bd67-7ff6-48a3-8b47-87cf6e0cebaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:00.133394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37bd67-7ff6-48a3-8b47-87cf6e0cebaa admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:00.133629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37bd67-7ff6-48a3-8b47-87cf6e0cebaa admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:00.133929 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37bd67-7ff6-48a3-8b47-87cf6e0cebaa admin admin] PATH_INFO: `/v3/projects/d88e08e98f054dddaa5caec1b0a39db4/users/dfbd9733da8247f797f5c80d0d2c0ceb/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:00.148550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37bd67-7ff6-48a3-8b47-87cf6e0cebaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d88e08e98f054dddaa5caec1b0a39db4, user_id=dfbd9733da8247f797f5c80d0d2c0ceb, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:00.151765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37bd67-7ff6-48a3-8b47-87cf6e0cebaa admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:00.164769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1008/2018] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:00 2022] PUT /identity/v3/projects/d88e08e98f054dddaa5caec1b0a39db4/users/dfbd9733da8247f797f5c80d0d2c0ceb/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:00.175509 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f697539-605f-437f-8ad2-8d831bd1f2c2 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:00.175807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f697539-605f-437f-8ad2-8d831bd1f2c2 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:00.176122 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f697539-605f-437f-8ad2-8d831bd1f2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:00.202615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-8f697539-605f-437f-8ad2-8d831bd1f2c2 None None] MFA Rules not processed for user `dfbd9733da8247f797f5c80d0d2c0ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:00.204151 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8f697539-605f-437f-8ad2-8d831bd1f2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:00.273358 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1011/2019] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:00.281305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37bd67-7ff6-48a3-8b47-87cf6e0cebaa admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:00.283007 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0f37bd67-7ff6-48a3-8b47-87cf6e0cebaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:00.340040 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37bd67-7ff6-48a3-8b47-87cf6e0cebaa admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:00.343700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3670bc1-f126-4f74-9007-6aa4804d2cd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:00.344373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3670bc1-f126-4f74-9007-6aa4804d2cd8 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:00.344661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3670bc1-f126-4f74-9007-6aa4804d2cd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:00.344939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3670bc1-f126-4f74-9007-6aa4804d2cd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:00.353460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3670bc1-f126-4f74-9007-6aa4804d2cd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:00.354911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3670bc1-f126-4f74-9007-6aa4804d2cd8 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:00.363612 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1009/2020] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:53:10.265313 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=85,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:10.359620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:34.333609 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f697539-605f-437f-8ad2-8d831bd1f2c2 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:34.333609 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8f697539-605f-437f-8ad2-8d831bd1f2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:34.403622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f697539-605f-437f-8ad2-8d831bd1f2c2 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:34.408353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc01c8-8649-470f-a856-6f693323db2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:34.409232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc01c8-8649-470f-a856-6f693323db2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:34.409515 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc01c8-8649-470f-a856-6f693323db2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:34.409797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc01c8-8649-470f-a856-6f693323db2a admin admin] PATH_INFO: `/v3/users/a825f40528e146dbb885bbf023be459b` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:34.410991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc01c8-8649-470f-a856-6f693323db2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a825f40528e146dbb885bbf023be459b)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:34.412987 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bc01c8-8649-470f-a856-6f693323db2a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:34.464141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-75bc01c8-8649-470f-a856-6f693323db2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a825f40528e146dbb885bbf023be459b'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:34.477615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-75bc01c8-8649-470f-a856-6f693323db2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a825f40528e146dbb885bbf023be459b'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:34.489985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-75bc01c8-8649-470f-a856-6f693323db2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a825f40528e146dbb885bbf023be459b'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:34.490665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-75bc01c8-8649-470f-a856-6f693323db2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a825f40528e146dbb885bbf023be459b'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:34.498506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-75bc01c8-8649-470f-a856-6f693323db2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a825f40528e146dbb885bbf023be459b'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:34.505999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1012/2021] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:53:34 2022] DELETE /identity/v3/users/a825f40528e146dbb885bbf023be459b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:34.510747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7b153-8571-486a-ba3d-f8f6d8bf7850 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:34.511582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7b153-8571-486a-ba3d-f8f6d8bf7850 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:34.511582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7b153-8571-486a-ba3d-f8f6d8bf7850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:34.529502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-7cc7b153-8571-486a-ba3d-f8f6d8bf7850 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:34.530932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7cc7b153-8571-486a-ba3d-f8f6d8bf7850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:34.593105 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1010/2022] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:34.602562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc01c8-8649-470f-a856-6f693323db2a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:34.604098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-75bc01c8-8649-470f-a856-6f693323db2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:34.669048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc01c8-8649-470f-a856-6f693323db2a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:34.673497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6d1b7-758e-4f6b-b796-61aa799df9ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:34.674250 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6d1b7-758e-4f6b-b796-61aa799df9ea service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:34.674609 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6d1b7-758e-4f6b-b796-61aa799df9ea service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:34.674963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6d1b7-758e-4f6b-b796-61aa799df9ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:34.684424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e6d1b7-758e-4f6b-b796-61aa799df9ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:34.685876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e6d1b7-758e-4f6b-b796-61aa799df9ea service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:34.697196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1013/2023] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:53:35.014889 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7b153-8571-486a-ba3d-f8f6d8bf7850 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:35.016934 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7cc7b153-8571-486a-ba3d-f8f6d8bf7850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:35.036124 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7b153-8571-486a-ba3d-f8f6d8bf7850 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:35.042380 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:35.051299 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:35.051590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:35.051874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] PATH_INFO: `/v3/projects/7f9bab2f74834f07b75d0b697068bfa8` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:35.053273 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f9bab2f74834f07b75d0b697068bfa8)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:35.055199 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:35.103720 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f9bab2f74834f07b75d0b697068bfa8', 'request_id': 'req-44eddb30-709d-4b6f-9537-416b92711020'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.111878 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f9bab2f74834f07b75d0b697068bfa8', 'request_id': 'req-44eddb30-709d-4b6f-9537-416b92711020'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.117621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f9bab2f74834f07b75d0b697068bfa8', 'request_id': 'req-44eddb30-709d-4b6f-9537-416b92711020'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.128916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] The token cache is being invalidate because project 7f9bab2f74834f07b75d0b697068bfa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:53:35.129209 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.129882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.131434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1011/2024] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:53:35 2022] DELETE /identity/v3/projects/7f9bab2f74834f07b75d0b697068bfa8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:35.135564 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6d1b7-758e-4f6b-b796-61aa799df9ea service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:35.146806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6d1b7-758e-4f6b-b796-61aa799df9ea service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:35.168034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:35.210071 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8bc83-e5af-49d8-b8b0-9ec4c26b79ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:35.211017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8bc83-e5af-49d8-b8b0-9ec4c26b79ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:35.211301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8bc83-e5af-49d8-b8b0-9ec4c26b79ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:35.211577 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8bc83-e5af-49d8-b8b0-9ec4c26b79ed admin admin] PATH_INFO: `/v3/users/377bc5f882714749bf5870df1703e691` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:35.212842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8bc83-e5af-49d8-b8b0-9ec4c26b79ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=377bc5f882714749bf5870df1703e691)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:35.214828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8bc83-e5af-49d8-b8b0-9ec4c26b79ed admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:35.266086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-16a8bc83-e5af-49d8-b8b0-9ec4c26b79ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '377bc5f882714749bf5870df1703e691'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.278719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-16a8bc83-e5af-49d8-b8b0-9ec4c26b79ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '377bc5f882714749bf5870df1703e691'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.286479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-16a8bc83-e5af-49d8-b8b0-9ec4c26b79ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '377bc5f882714749bf5870df1703e691'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.287650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-16a8bc83-e5af-49d8-b8b0-9ec4c26b79ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '377bc5f882714749bf5870df1703e691'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.295092 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-16a8bc83-e5af-49d8-b8b0-9ec4c26b79ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '377bc5f882714749bf5870df1703e691'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.302014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1014/2025] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:53:35 2022] DELETE /identity/v3/users/377bc5f882714749bf5870df1703e691 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:35.579088 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:35.580571 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:35.598988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eddb30-709d-4b6f-9537-416b92711020 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:35.670199 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e77ef1-d2f3-427b-91c3-26ae861ea58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:35.671587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e77ef1-d2f3-427b-91c3-26ae861ea58d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:35.671881 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e77ef1-d2f3-427b-91c3-26ae861ea58d admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:35.672208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e77ef1-d2f3-427b-91c3-26ae861ea58d admin admin] PATH_INFO: `/v3/projects/fedd4b712f394cd1b74ae72a51e91aa9` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:35.673554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e77ef1-d2f3-427b-91c3-26ae861ea58d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fedd4b712f394cd1b74ae72a51e91aa9)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:35.675466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e77ef1-d2f3-427b-91c3-26ae861ea58d admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:35.731611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:35.752738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-08e77ef1-d2f3-427b-91c3-26ae861ea58d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fedd4b712f394cd1b74ae72a51e91aa9', 'request_id': 'req-08e77ef1-d2f3-427b-91c3-26ae861ea58d'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.760560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-08e77ef1-d2f3-427b-91c3-26ae861ea58d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fedd4b712f394cd1b74ae72a51e91aa9', 'request_id': 'req-08e77ef1-d2f3-427b-91c3-26ae861ea58d'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.769271 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-08e77ef1-d2f3-427b-91c3-26ae861ea58d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fedd4b712f394cd1b74ae72a51e91aa9', 'request_id': 'req-08e77ef1-d2f3-427b-91c3-26ae861ea58d'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.781320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-08e77ef1-d2f3-427b-91c3-26ae861ea58d admin admin] The token cache is being invalidate because project fedd4b712f394cd1b74ae72a51e91aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:53:35.781552 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-08e77ef1-d2f3-427b-91c3-26ae861ea58d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.783563 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-08e77ef1-d2f3-427b-91c3-26ae861ea58d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:35.785253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1012/2026] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:53:35 2022] DELETE /identity/v3/projects/fedd4b712f394cd1b74ae72a51e91aa9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:35.790440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fde15-5959-4a4b-9d78-10a3b8016549 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:35.790993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fde15-5959-4a4b-9d78-10a3b8016549 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:35.791110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fde15-5959-4a4b-9d78-10a3b8016549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:35.812662 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-d64fde15-5959-4a4b-9d78-10a3b8016549 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:35.813888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d64fde15-5959-4a4b-9d78-10a3b8016549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:35.886366 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1015/2027] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:53:35.895727 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d78c3-a41c-4385-a40c-82f808e9f768 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:35.896561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d78c3-a41c-4385-a40c-82f808e9f768 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:35.897023 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d78c3-a41c-4385-a40c-82f808e9f768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:35.918025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-7a8d78c3-a41c-4385-a40c-82f808e9f768 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:35.919248 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7a8d78c3-a41c-4385-a40c-82f808e9f768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:35.927802 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1013/2028] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:35.932253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64fde15-5959-4a4b-9d78-10a3b8016549 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:35.942489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64fde15-5959-4a4b-9d78-10a3b8016549 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:35.947560 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c65c18-4ed1-4bb0-8288-978fdb3323e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:35.948271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c65c18-4ed1-4bb0-8288-978fdb3323e6 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:35.948687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c65c18-4ed1-4bb0-8288-978fdb3323e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:35.948976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c65c18-4ed1-4bb0-8288-978fdb3323e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:35.949712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c65c18-4ed1-4bb0-8288-978fdb3323e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:53:35.950025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c65c18-4ed1-4bb0-8288-978fdb3323e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:35.951821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c65c18-4ed1-4bb0-8288-978fdb3323e6 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:35.968430 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1016/2029] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:53:35.970466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d78c3-a41c-4385-a40c-82f808e9f768 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:35.980225 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d78c3-a41c-4385-a40c-82f808e9f768 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:35.985456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c717f9a-a0e9-4e69-8ab3-532786751f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:35.986248 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c717f9a-a0e9-4e69-8ab3-532786751f97 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:35.986535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c717f9a-a0e9-4e69-8ab3-532786751f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:35.986834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c717f9a-a0e9-4e69-8ab3-532786751f97 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:35.987607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c717f9a-a0e9-4e69-8ab3-532786751f97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:35.989667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c717f9a-a0e9-4e69-8ab3-532786751f97 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:36.016437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1014/2030] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:53:35 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:36.021036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c65c18-4ed1-4bb0-8288-978fdb3323e6 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:36.032061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c65c18-4ed1-4bb0-8288-978fdb3323e6 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:36.058719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:36.090828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983c912-aa61-4175-9c7d-5121397f4cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:36.091518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983c912-aa61-4175-9c7d-5121397f4cd2 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:36.091804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983c912-aa61-4175-9c7d-5121397f4cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:36.092087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c717f9a-a0e9-4e69-8ab3-532786751f97 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:36.097969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1983c912-aa61-4175-9c7d-5121397f4cd2 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:36.097969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983c912-aa61-4175-9c7d-5121397f4cd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:36.097969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1983c912-aa61-4175-9c7d-5121397f4cd2 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:36.101469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4c717f9a-a0e9-4e69-8ab3-532786751f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:36.119962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c717f9a-a0e9-4e69-8ab3-532786751f97 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:36.124394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1017/2031] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:53:36 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:53:36.125318 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983c912-aa61-4175-9c7d-5121397f4cd2 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:36.125439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd447c-e077-4688-9036-0120eff099e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:36.126264 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd447c-e077-4688-9036-0120eff099e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:36.126551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd447c-e077-4688-9036-0120eff099e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:36.126828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd447c-e077-4688-9036-0120eff099e2 admin admin] PATH_INFO: `/v3/users/bbafbab4281e4a6eb168aa8e3ea151c2` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:36.128081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd447c-e077-4688-9036-0120eff099e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbafbab4281e4a6eb168aa8e3ea151c2)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:36.130010 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd447c-e077-4688-9036-0120eff099e2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:36.135215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1983c912-aa61-4175-9c7d-5121397f4cd2 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:36.140927 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2710751-0582-4c89-aa93-6d91189956d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:36.141774 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2710751-0582-4c89-aa93-6d91189956d1 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:36.142224 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2710751-0582-4c89-aa93-6d91189956d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:36.142513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2710751-0582-4c89-aa93-6d91189956d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:36.143255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2710751-0582-4c89-aa93-6d91189956d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:36.144885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2710751-0582-4c89-aa93-6d91189956d1 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:36.155193 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1018/2032] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:53:36.160421 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2710751-0582-4c89-aa93-6d91189956d1 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:36.171028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2710751-0582-4c89-aa93-6d91189956d1 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:36.175780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6479ac6-ae59-45d4-8535-d5c441a5a7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:36.176480 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6479ac6-ae59-45d4-8535-d5c441a5a7ff admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:36.176770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6479ac6-ae59-45d4-8535-d5c441a5a7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:36.177054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6479ac6-ae59-45d4-8535-d5c441a5a7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:36.177781 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6479ac6-ae59-45d4-8535-d5c441a5a7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:36.179368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6479ac6-ae59-45d4-8535-d5c441a5a7ff admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:36.181456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-22bd447c-e077-4688-9036-0120eff099e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbafbab4281e4a6eb168aa8e3ea151c2'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:36.190932 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1019/2033] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:53:36.194596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-22bd447c-e077-4688-9036-0120eff099e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbafbab4281e4a6eb168aa8e3ea151c2'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:36.195321 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6479ac6-ae59-45d4-8535-d5c441a5a7ff admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:36.195457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-22bd447c-e077-4688-9036-0120eff099e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbafbab4281e4a6eb168aa8e3ea151c2'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:36.197001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b6479ac6-ae59-45d4-8535-d5c441a5a7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:36.204236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-22bd447c-e077-4688-9036-0120eff099e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbafbab4281e4a6eb168aa8e3ea151c2'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:36.204903 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-22bd447c-e077-4688-9036-0120eff099e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbafbab4281e4a6eb168aa8e3ea151c2'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:36.209996 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6479ac6-ae59-45d4-8535-d5c441a5a7ff admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:36.214586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689ee24-80e6-401f-9d09-26f16d8fbeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:36.215466 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689ee24-80e6-401f-9d09-26f16d8fbeec admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:36.215746 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689ee24-80e6-401f-9d09-26f16d8fbeec admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:36.216064 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689ee24-80e6-401f-9d09-26f16d8fbeec admin admin] PATH_INFO: `/v3/projects/5a29e7443fe84c98a77ab817dccc51a2/users/7018b8fac644448299c2dff4158134da/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:36.218978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1015/2034] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:53:36 2022] DELETE /identity/v3/users/bbafbab4281e4a6eb168aa8e3ea151c2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:53:36.223277 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4273d-e554-418d-b669-b724a95e1e99 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:36.223566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4273d-e554-418d-b669-b724a95e1e99 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:36.223847 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b4273d-e554-418d-b669-b724a95e1e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:36.235348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689ee24-80e6-401f-9d09-26f16d8fbeec admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a29e7443fe84c98a77ab817dccc51a2, user_id=7018b8fac644448299c2dff4158134da, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:36.238034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689ee24-80e6-401f-9d09-26f16d8fbeec admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:36.241001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-a9b4273d-e554-418d-b669-b724a95e1e99 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:36.242257 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a9b4273d-e554-418d-b669-b724a95e1e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:36.252107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1020/2035] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:36 2022] PUT /identity/v3/projects/5a29e7443fe84c98a77ab817dccc51a2/users/7018b8fac644448299c2dff4158134da/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:53:36.262325 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192babe-3d0e-48af-aca7-52ad19fe7840 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:36.262611 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192babe-3d0e-48af-aca7-52ad19fe7840 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:36.262889 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4192babe-3d0e-48af-aca7-52ad19fe7840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:36.289334 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-4192babe-3d0e-48af-aca7-52ad19fe7840 None None] MFA Rules not processed for user `7018b8fac644448299c2dff4158134da`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:36.290561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4192babe-3d0e-48af-aca7-52ad19fe7840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:36.294984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:36.310851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1016/2036] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:53:36.320626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4273d-e554-418d-b669-b724a95e1e99 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:36.322216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a9b4273d-e554-418d-b669-b724a95e1e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:36.366963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1021/2037] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:36.371590 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192babe-3d0e-48af-aca7-52ad19fe7840 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:36.383408 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b4273d-e554-418d-b669-b724a95e1e99 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:36.447373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318fc1af-3682-465c-b225-92ec294b9435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:36.448672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fc1af-3682-465c-b225-92ec294b9435 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:36.448963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fc1af-3682-465c-b225-92ec294b9435 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:36.449242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318fc1af-3682-465c-b225-92ec294b9435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:36.503254 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4192babe-3d0e-48af-aca7-52ad19fe7840 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:36.514334 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104bf79-e1c6-4f45-be82-705899d1796c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:36.519768 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104bf79-e1c6-4f45-be82-705899d1796c admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:36.520368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104bf79-e1c6-4f45-be82-705899d1796c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:36.520869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5104bf79-e1c6-4f45-be82-705899d1796c admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:36.522047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104bf79-e1c6-4f45-be82-705899d1796c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:36.524235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5104bf79-e1c6-4f45-be82-705899d1796c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:36.541252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318fc1af-3682-465c-b225-92ec294b9435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:36.542727 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318fc1af-3682-465c-b225-92ec294b9435 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:36.556940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:36.572679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1017/2038] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:53:36.586482 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1022/2039] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:53:36 2022] POST /identity/v3/projects => generated 385 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:36.594656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318fc1af-3682-465c-b225-92ec294b9435 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:36.605213 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318fc1af-3682-465c-b225-92ec294b9435 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:36.683673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304e35e-4bf3-4fba-864e-191d5bf0fce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:36.684430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304e35e-4bf3-4fba-864e-191d5bf0fce3 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:36.685968 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304e35e-4bf3-4fba-864e-191d5bf0fce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:36.686353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304e35e-4bf3-4fba-864e-191d5bf0fce3 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:36.688539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304e35e-4bf3-4fba-864e-191d5bf0fce3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:36.694423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304e35e-4bf3-4fba-864e-191d5bf0fce3 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:36.716034 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:36.739403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1018/2040] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:53:36 2022] POST /identity/v3/users => generated 299 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:36.743344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104bf79-e1c6-4f45-be82-705899d1796c admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:36.761456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5104bf79-e1c6-4f45-be82-705899d1796c admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:36.771252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce26795-df15-453c-89ec-6e7d233944bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:36.772153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce26795-df15-453c-89ec-6e7d233944bb admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:36.772721 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce26795-df15-453c-89ec-6e7d233944bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:36.773242 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce26795-df15-453c-89ec-6e7d233944bb admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:36.774827 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce26795-df15-453c-89ec-6e7d233944bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:36.776867 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce26795-df15-453c-89ec-6e7d233944bb admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:36.795727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1023/2041] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:53:36.798487 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304e35e-4bf3-4fba-864e-191d5bf0fce3 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:36.819589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304e35e-4bf3-4fba-864e-191d5bf0fce3 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:36.826044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1c3f4-0837-4ace-ab90-fb1191f38f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:36.827866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1c3f4-0837-4ace-ab90-fb1191f38f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:36.828284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1c3f4-0837-4ace-ab90-fb1191f38f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:36.828919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1c3f4-0837-4ace-ab90-fb1191f38f91 admin admin] PATH_INFO: `/v3/projects/0e5ff4cff4fd48c6a1964d76e974ba0a/users/939b162e4fc14257942d978974821b33/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:36.850761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1c3f4-0837-4ace-ab90-fb1191f38f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5ff4cff4fd48c6a1964d76e974ba0a, user_id=939b162e4fc14257942d978974821b33, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:36.855826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1c3f4-0837-4ace-ab90-fb1191f38f91 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:36.893073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1019/2042] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:36 2022] PUT /identity/v3/projects/0e5ff4cff4fd48c6a1964d76e974ba0a/users/939b162e4fc14257942d978974821b33/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:36.898132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce26795-df15-453c-89ec-6e7d233944bb admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:36.913483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce26795-df15-453c-89ec-6e7d233944bb admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:37.119988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1c3f4-0837-4ace-ab90-fb1191f38f91 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:37.121793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a4a1c3f4-0837-4ace-ab90-fb1191f38f91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:37.132157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7731e46-ccb5-4613-982e-d9a5c3c4f162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:37.132513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7731e46-ccb5-4613-982e-d9a5c3c4f162 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:37.132753 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7731e46-ccb5-4613-982e-d9a5c3c4f162 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:37.133031 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7731e46-ccb5-4613-982e-d9a5c3c4f162 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:37.133830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7731e46-ccb5-4613-982e-d9a5c3c4f162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:37.136811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7731e46-ccb5-4613-982e-d9a5c3c4f162 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:37.142119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1c3f4-0837-4ace-ab90-fb1191f38f91 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:37.166271 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:37.167198 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:37.167461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:37.167736 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] PATH_INFO: `/v3/projects/2c7a2c14a1cc465896a924f2ebab809e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:37.171010 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c7a2c14a1cc465896a924f2ebab809e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:37.173555 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:37.180017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1024/2043] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:36 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:53:37.181660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7731e46-ccb5-4613-982e-d9a5c3c4f162 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:37.200805 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7731e46-ccb5-4613-982e-d9a5c3c4f162 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:37.210791 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235e729-b475-4172-a259-748fd14c4dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:37.211803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235e729-b475-4172-a259-748fd14c4dec admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:37.212225 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235e729-b475-4172-a259-748fd14c4dec admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:37.212539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235e729-b475-4172-a259-748fd14c4dec admin admin] PATH_INFO: `/v3/projects/0e5ff4cff4fd48c6a1964d76e974ba0a/users/939b162e4fc14257942d978974821b33/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:37.214739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235e729-b475-4172-a259-748fd14c4dec admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5ff4cff4fd48c6a1964d76e974ba0a, user_id=939b162e4fc14257942d978974821b33, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:37.217821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235e729-b475-4172-a259-748fd14c4dec admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:37.234239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1025/2044] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:37 2022] PUT /identity/v3/projects/0e5ff4cff4fd48c6a1964d76e974ba0a/users/939b162e4fc14257942d978974821b33/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:37.244185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c7a2c14a1cc465896a924f2ebab809e', 'request_id': 'req-140ab2de-4039-4458-83ed-cb32971c3dc6'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:37.245399 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca7342-7b63-4e28-a1ca-188fe737feab None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:37.245693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca7342-7b63-4e28-a1ca-188fe737feab None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:37.246028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca7342-7b63-4e28-a1ca-188fe737feab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:37.253987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c7a2c14a1cc465896a924f2ebab809e', 'request_id': 'req-140ab2de-4039-4458-83ed-cb32971c3dc6'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:37.265461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c7a2c14a1cc465896a924f2ebab809e', 'request_id': 'req-140ab2de-4039-4458-83ed-cb32971c3dc6'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:37.279257 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] The token cache is being invalidate because project 2c7a2c14a1cc465896a924f2ebab809e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:53:37.279546 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:37.280075 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-caca7342-7b63-4e28-a1ca-188fe737feab None None] MFA Rules not processed for user `939b162e4fc14257942d978974821b33`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:37.280253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:37.281279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-caca7342-7b63-4e28-a1ca-188fe737feab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:37.284878 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1020/2045] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:53:37 2022] DELETE /identity/v3/projects/2c7a2c14a1cc465896a924f2ebab809e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:37.285631 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:37.286996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:37.368046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1026/2046] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:37 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:53:37.368249 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ab2de-4039-4458-83ed-cb32971c3dc6 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:37.373991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a4cf5-718a-4528-9bc5-66dc80b51c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:37.375739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a4cf5-718a-4528-9bc5-66dc80b51c0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:37.376057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a4cf5-718a-4528-9bc5-66dc80b51c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:37.376339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21a4cf5-718a-4528-9bc5-66dc80b51c0c admin admin] PATH_INFO: `/v3/users/2229a34f5b5648808850c6bcc09a8860` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:37.377605 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a4cf5-718a-4528-9bc5-66dc80b51c0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2229a34f5b5648808850c6bcc09a8860)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:37.379604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21a4cf5-718a-4528-9bc5-66dc80b51c0c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:37.387632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca7342-7b63-4e28-a1ca-188fe737feab None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:37.389353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-caca7342-7b63-4e28-a1ca-188fe737feab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:37.439164 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c21a4cf5-718a-4528-9bc5-66dc80b51c0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2229a34f5b5648808850c6bcc09a8860'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:37.445255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:37.449453 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:INSERT=3,keystone:SELECT=82,keystone:DELETE=15 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:37.459647 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c21a4cf5-718a-4528-9bc5-66dc80b51c0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2229a34f5b5648808850c6bcc09a8860'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:37.460809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c21a4cf5-718a-4528-9bc5-66dc80b51c0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2229a34f5b5648808850c6bcc09a8860'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:37.462735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caca7342-7b63-4e28-a1ca-188fe737feab None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:37.467400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed55f8f-131c-47ba-970a-7d0067b6766d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:37.468132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed55f8f-131c-47ba-970a-7d0067b6766d service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:37.468415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed55f8f-131c-47ba-970a-7d0067b6766d service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:37.468788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed55f8f-131c-47ba-970a-7d0067b6766d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:37.469143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c21a4cf5-718a-4528-9bc5-66dc80b51c0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2229a34f5b5648808850c6bcc09a8860'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:37.470063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c21a4cf5-718a-4528-9bc5-66dc80b51c0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2229a34f5b5648808850c6bcc09a8860'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:37.470672 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7ed55f8f-131c-47ba-970a-7d0067b6766d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:37.485820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed55f8f-131c-47ba-970a-7d0067b6766d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:37.487322 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed55f8f-131c-47ba-970a-7d0067b6766d service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:37.491195 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1021/2047] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:53:37 2022] DELETE /identity/v3/users/2229a34f5b5648808850c6bcc09a8860 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:53:37.498275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1027/2048] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:37 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:53:37.966170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a4cf5-718a-4528-9bc5-66dc80b51c0c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:37.967691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c21a4cf5-718a-4528-9bc5-66dc80b51c0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:38.141384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21a4cf5-718a-4528-9bc5-66dc80b51c0c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:38.153799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:38.153799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:38.153799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:38.153799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] PATH_INFO: `/v3/projects/1d71574c4e51413e989675de41cf398d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:38.154133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d71574c4e51413e989675de41cf398d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:38.154133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:38.258379 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d71574c4e51413e989675de41cf398d', 'request_id': 'req-1f67e3ea-4daf-4596-a9c6-c102e579e44a'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:38.269234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d71574c4e51413e989675de41cf398d', 'request_id': 'req-1f67e3ea-4daf-4596-a9c6-c102e579e44a'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:38.282266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d71574c4e51413e989675de41cf398d', 'request_id': 'req-1f67e3ea-4daf-4596-a9c6-c102e579e44a'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:38.309568 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] The token cache is being invalidate because project 1d71574c4e51413e989675de41cf398d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:53:38.310340 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:38.311297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:38.314005 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1022/2049] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:53:37 2022] DELETE /identity/v3/projects/1d71574c4e51413e989675de41cf398d => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:38.320070 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407260fc-495a-4c57-910c-595614744cd9 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:38.321621 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407260fc-495a-4c57-910c-595614744cd9 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:38.326890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407260fc-495a-4c57-910c-595614744cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:38.387020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-407260fc-495a-4c57-910c-595614744cd9 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:38.391216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-407260fc-495a-4c57-910c-595614744cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:38.521522 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:38.522274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:38.538371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1028/2050] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:38.549489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306f054-c0bb-48bb-8ecd-cb5aa51ecb0d None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:38.551276 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306f054-c0bb-48bb-8ecd-cb5aa51ecb0d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:38.551276 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306f054-c0bb-48bb-8ecd-cb5aa51ecb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:38.570185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67e3ea-4daf-4596-a9c6-c102e579e44a admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:38.585641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f4689-280a-45de-a055-7571467b14d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:38.588696 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f4689-280a-45de-a055-7571467b14d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:38.602348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-7306f054-c0bb-48bb-8ecd-cb5aa51ecb0d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:38.603221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f4689-280a-45de-a055-7571467b14d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:38.603803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f4689-280a-45de-a055-7571467b14d4 admin admin] PATH_INFO: `/v3/users/7018b8fac644448299c2dff4158134da` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:38.605828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f4689-280a-45de-a055-7571467b14d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7018b8fac644448299c2dff4158134da)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:38.608011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f4689-280a-45de-a055-7571467b14d4 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:38.608201 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7306f054-c0bb-48bb-8ecd-cb5aa51ecb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:38.646969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1029/2051] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:38.650974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306f054-c0bb-48bb-8ecd-cb5aa51ecb0d None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:38.679870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306f054-c0bb-48bb-8ecd-cb5aa51ecb0d None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:38.701027 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae41666-b6f5-4056-a728-7d1a563c2716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:38.701713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae41666-b6f5-4056-a728-7d1a563c2716 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:38.702001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae41666-b6f5-4056-a728-7d1a563c2716 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:38.702277 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae41666-b6f5-4056-a728-7d1a563c2716 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:38.703057 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae41666-b6f5-4056-a728-7d1a563c2716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:53:38.703363 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae41666-b6f5-4056-a728-7d1a563c2716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:38.710099 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae41666-b6f5-4056-a728-7d1a563c2716 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:38.720937 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d65f4689-280a-45de-a055-7571467b14d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7018b8fac644448299c2dff4158134da'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:38.737263 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:38.743398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1030/2052] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:53:38.750611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d65f4689-280a-45de-a055-7571467b14d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7018b8fac644448299c2dff4158134da'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:38.750797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae41666-b6f5-4056-a728-7d1a563c2716 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:38.752955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7ae41666-b6f5-4056-a728-7d1a563c2716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:38.753100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d65f4689-280a-45de-a055-7571467b14d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7018b8fac644448299c2dff4158134da'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:38.775476 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d65f4689-280a-45de-a055-7571467b14d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7018b8fac644448299c2dff4158134da'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:38.776453 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d65f4689-280a-45de-a055-7571467b14d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7018b8fac644448299c2dff4158134da'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:38.777113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae41666-b6f5-4056-a728-7d1a563c2716 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:38.785443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a9abf-cdc2-4cc4-8e5d-7a7ebdb68459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:38.786215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a9abf-cdc2-4cc4-8e5d-7a7ebdb68459 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:38.786849 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a9abf-cdc2-4cc4-8e5d-7a7ebdb68459 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:38.787222 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a9abf-cdc2-4cc4-8e5d-7a7ebdb68459 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:38.789508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a9abf-cdc2-4cc4-8e5d-7a7ebdb68459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:38.793277 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a9abf-cdc2-4cc4-8e5d-7a7ebdb68459 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:38.803491 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1023/2053] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:53:38 2022] DELETE /identity/v3/users/7018b8fac644448299c2dff4158134da => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:38.807679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6164e-8bf6-4929-8f81-31a8f3a79685 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:38.809244 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6164e-8bf6-4929-8f81-31a8f3a79685 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:38.809513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6164e-8bf6-4929-8f81-31a8f3a79685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:38.838643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-78b6164e-8bf6-4929-8f81-31a8f3a79685 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:38.840161 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-78b6164e-8bf6-4929-8f81-31a8f3a79685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:38.850911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1031/2054] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:53:38 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:38.855041 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a9abf-cdc2-4cc4-8e5d-7a7ebdb68459 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:38.935843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1024/2055] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:53:38.945253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a9abf-cdc2-4cc4-8e5d-7a7ebdb68459 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:38.952037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69bb3c-747f-48e6-a146-f81bbb777c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:38.953721 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69bb3c-747f-48e6-a146-f81bbb777c8e admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:38.954349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69bb3c-747f-48e6-a146-f81bbb777c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:38.955545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69bb3c-747f-48e6-a146-f81bbb777c8e admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:38.956483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b69bb3c-747f-48e6-a146-f81bbb777c8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:38.959993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b69bb3c-747f-48e6-a146-f81bbb777c8e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:38.962763 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6164e-8bf6-4929-8f81-31a8f3a79685 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:38.965635 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-78b6164e-8bf6-4929-8f81-31a8f3a79685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:38.992358 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1032/2056] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:53:38 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:53:38.996029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69bb3c-747f-48e6-a146-f81bbb777c8e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:39.015494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69bb3c-747f-48e6-a146-f81bbb777c8e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:39.028178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48dedb-ee2e-4f52-9445-a26bd6f7d2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:39.028946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48dedb-ee2e-4f52-9445-a26bd6f7d2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:39.029306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48dedb-ee2e-4f52-9445-a26bd6f7d2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:39.029643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f48dedb-ee2e-4f52-9445-a26bd6f7d2d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:39.030456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48dedb-ee2e-4f52-9445-a26bd6f7d2d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:39.032874 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f48dedb-ee2e-4f52-9445-a26bd6f7d2d5 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:39.049418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1033/2057] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:53:39.050634 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48dedb-ee2e-4f52-9445-a26bd6f7d2d5 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:39.053084 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6164e-8bf6-4929-8f81-31a8f3a79685 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:39.070461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f48dedb-ee2e-4f52-9445-a26bd6f7d2d5 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:39.075463 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a4435-9737-4694-93fe-355369a9bb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:39.076180 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a4435-9737-4694-93fe-355369a9bb4e admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:39.076464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a4435-9737-4694-93fe-355369a9bb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:39.076741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377a4435-9737-4694-93fe-355369a9bb4e admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:39.077497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a4435-9737-4694-93fe-355369a9bb4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:39.079099 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377a4435-9737-4694-93fe-355369a9bb4e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:39.092511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1034/2058] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:53:39.096543 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a4435-9737-4694-93fe-355369a9bb4e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:39.107649 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377a4435-9737-4694-93fe-355369a9bb4e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:39.113629 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c389e-1d5d-41fa-b4ee-23002fa58565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:39.114888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c389e-1d5d-41fa-b4ee-23002fa58565 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:39.115248 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c389e-1d5d-41fa-b4ee-23002fa58565 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:39.115598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c389e-1d5d-41fa-b4ee-23002fa58565 admin admin] PATH_INFO: `/v3/projects/94b7f5f6eae14ba8b3854ed4d0fe24d7/users/5b2bdb85b2144a04bbc5aeb4998a09e4/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:39.116915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d8898-a83c-4f11-be95-c0d18c327371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:39.117674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d8898-a83c-4f11-be95-c0d18c327371 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:39.118031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d8898-a83c-4f11-be95-c0d18c327371 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:39.118566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d8898-a83c-4f11-be95-c0d18c327371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:39.135892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:39.138831 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d8898-a83c-4f11-be95-c0d18c327371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:39.140092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d8898-a83c-4f11-be95-c0d18c327371 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:39.148222 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c389e-1d5d-41fa-b4ee-23002fa58565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94b7f5f6eae14ba8b3854ed4d0fe24d7, user_id=5b2bdb85b2144a04bbc5aeb4998a09e4, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:39.152048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c389e-1d5d-41fa-b4ee-23002fa58565 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:39.153310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1025/2059] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:53:39.183026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1035/2060] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:39 2022] PUT /identity/v3/projects/94b7f5f6eae14ba8b3854ed4d0fe24d7/users/5b2bdb85b2144a04bbc5aeb4998a09e4/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:39.195907 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0571e2-fc98-4299-9c70-03a1fc641128 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:39.196425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0571e2-fc98-4299-9c70-03a1fc641128 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:39.197143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0571e2-fc98-4299-9c70-03a1fc641128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:39.231271 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4d0571e2-fc98-4299-9c70-03a1fc641128 None None] MFA Rules not processed for user `5b2bdb85b2144a04bbc5aeb4998a09e4`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:39.233289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4d0571e2-fc98-4299-9c70-03a1fc641128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:39.263091 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:39.321297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1026/2061] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:39.322972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c389e-1d5d-41fa-b4ee-23002fa58565 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:39.417717 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c389e-1d5d-41fa-b4ee-23002fa58565 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:39.423284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b371d0-ecdb-4833-9c62-e405d7313a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:39.424125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b371d0-ecdb-4833-9c62-e405d7313a47 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:39.424522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b371d0-ecdb-4833-9c62-e405d7313a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:39.424905 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b371d0-ecdb-4833-9c62-e405d7313a47 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:39.425728 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b371d0-ecdb-4833-9c62-e405d7313a47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:39.427755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b371d0-ecdb-4833-9c62-e405d7313a47 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:39.469025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1036/2062] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:53:39 2022] POST /identity/v3/projects => generated 375 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:39.470034 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0571e2-fc98-4299-9c70-03a1fc641128 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:39.475533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b371d0-ecdb-4833-9c62-e405d7313a47 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:39.484930 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d0b371d0-ecdb-4833-9c62-e405d7313a47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:39.485710 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0571e2-fc98-4299-9c70-03a1fc641128 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:39.555302 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b371d0-ecdb-4833-9c62-e405d7313a47 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:39.560550 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:39.560698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dba6ee-3609-42c9-9214-bf3e20a9d648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:39.561288 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dba6ee-3609-42c9-9214-bf3e20a9d648 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:39.561812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:39.562050 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dba6ee-3609-42c9-9214-bf3e20a9d648 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:39.562345 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dba6ee-3609-42c9-9214-bf3e20a9d648 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:39.562455 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:39.562840 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] PATH_INFO: `/v3/projects/5a29e7443fe84c98a77ab817dccc51a2` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:39.563047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dba6ee-3609-42c9-9214-bf3e20a9d648 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:39.564280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a29e7443fe84c98a77ab817dccc51a2)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:39.565478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dba6ee-3609-42c9-9214-bf3e20a9d648 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:39.566393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:39.588221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1027/2063] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:53:39 2022] POST /identity/v3/users => generated 288 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:39.593850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dba6ee-3609-42c9-9214-bf3e20a9d648 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:39.608046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dba6ee-3609-42c9-9214-bf3e20a9d648 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:39.613694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27c281-0d79-4b3d-a9e2-317374a097df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:39.614496 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27c281-0d79-4b3d-a9e2-317374a097df admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:39.615002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27c281-0d79-4b3d-a9e2-317374a097df admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:39.615281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27c281-0d79-4b3d-a9e2-317374a097df admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:39.616198 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27c281-0d79-4b3d-a9e2-317374a097df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:39.618552 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27c281-0d79-4b3d-a9e2-317374a097df admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:39.632391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1028/2064] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:53:39.635111 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27c281-0d79-4b3d-a9e2-317374a097df admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:39.636768 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a29e7443fe84c98a77ab817dccc51a2', 'request_id': 'req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:39.650306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27c281-0d79-4b3d-a9e2-317374a097df admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:39.657835 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a29e7443fe84c98a77ab817dccc51a2', 'request_id': 'req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:39.670275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a29e7443fe84c98a77ab817dccc51a2', 'request_id': 'req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:39.696269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] The token cache is being invalidate because project 5a29e7443fe84c98a77ab817dccc51a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:53:39.696562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:39.697246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:39.698992 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1037/2065] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:53:39 2022] DELETE /identity/v3/projects/5a29e7443fe84c98a77ab817dccc51a2 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:39.703542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:39.705137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:39.712312 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:39.738240 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f01ad3-cdba-466d-9a85-cdf60cdfdf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:39.738918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f01ad3-cdba-466d-9a85-cdf60cdfdf19 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:39.739202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f01ad3-cdba-466d-9a85-cdf60cdfdf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:39.739482 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f01ad3-cdba-466d-9a85-cdf60cdfdf19 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:39.740364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f01ad3-cdba-466d-9a85-cdf60cdfdf19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:39.741973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f01ad3-cdba-466d-9a85-cdf60cdfdf19 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:39.755618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1029/2066] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:39 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:53:39.757234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f01ad3-cdba-466d-9a85-cdf60cdfdf19 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:39.759301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-30f01ad3-cdba-466d-9a85-cdf60cdfdf19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:39.777354 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f01ad3-cdba-466d-9a85-cdf60cdfdf19 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:39.785671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d287122c-d874-4add-b604-3f9cc9d24021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:39.786585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287122c-d874-4add-b604-3f9cc9d24021 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:39.786873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287122c-d874-4add-b604-3f9cc9d24021 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:39.787148 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d287122c-d874-4add-b604-3f9cc9d24021 admin admin] PATH_INFO: `/v3/projects/14f0a4078b83478890ab8192f7cecaea/users/a6715f7f5878486c92f292f686f2b17f/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:39.798978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a924ea-10bc-491a-9fd6-3e2e5a3fe78e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:39.805041 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87ac23-f346-42b1-bef7-08a073d8f183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:39.806361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87ac23-f346-42b1-bef7-08a073d8f183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:39.806727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87ac23-f346-42b1-bef7-08a073d8f183 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:39.807082 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87ac23-f346-42b1-bef7-08a073d8f183 admin admin] PATH_INFO: `/v3/users/939b162e4fc14257942d978974821b33` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:39.808661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87ac23-f346-42b1-bef7-08a073d8f183 admin admin] RBAC: Authorizing `identity:delete_user(user_id=939b162e4fc14257942d978974821b33)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:39.811458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f87ac23-f346-42b1-bef7-08a073d8f183 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:39.813176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287122c-d874-4add-b604-3f9cc9d24021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f0a4078b83478890ab8192f7cecaea, user_id=a6715f7f5878486c92f292f686f2b17f, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:39.816117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d287122c-d874-4add-b604-3f9cc9d24021 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:39.837940 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1030/2067] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:39 2022] PUT /identity/v3/projects/14f0a4078b83478890ab8192f7cecaea/users/a6715f7f5878486c92f292f686f2b17f/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:53:39.847525 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675e3e0-359e-46f9-a173-ca8b59e7ad38 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:39.848917 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675e3e0-359e-46f9-a173-ca8b59e7ad38 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:39.849285 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675e3e0-359e-46f9-a173-ca8b59e7ad38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:39.877563 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:39.881767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-7675e3e0-359e-46f9-a173-ca8b59e7ad38 None None] MFA Rules not processed for user `a6715f7f5878486c92f292f686f2b17f`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:39.883872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7675e3e0-359e-46f9-a173-ca8b59e7ad38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:39.884763 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4f87ac23-f346-42b1-bef7-08a073d8f183 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939b162e4fc14257942d978974821b33'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:39.908963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4f87ac23-f346-42b1-bef7-08a073d8f183 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939b162e4fc14257942d978974821b33'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:39.930158 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4f87ac23-f346-42b1-bef7-08a073d8f183 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939b162e4fc14257942d978974821b33'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:39.931020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4f87ac23-f346-42b1-bef7-08a073d8f183 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939b162e4fc14257942d978974821b33'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:39.943474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4f87ac23-f346-42b1-bef7-08a073d8f183 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939b162e4fc14257942d978974821b33'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:39.954651 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1038/2068] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:53:39 2022] DELETE /identity/v3/users/939b162e4fc14257942d978974821b33 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:53:39.974348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1031/2069] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:39.978924 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87ac23-f346-42b1-bef7-08a073d8f183 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:39.980430 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4f87ac23-f346-42b1-bef7-08a073d8f183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:40.052955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87ac23-f346-42b1-bef7-08a073d8f183 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:40.059651 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0569fe6-4b7f-47b8-bc98-e2ff68572c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:40.061970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0569fe6-4b7f-47b8-bc98-e2ff68572c1b admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:40.062305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0569fe6-4b7f-47b8-bc98-e2ff68572c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:40.062617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0569fe6-4b7f-47b8-bc98-e2ff68572c1b admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:40.063347 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0569fe6-4b7f-47b8-bc98-e2ff68572c1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:40.065338 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0569fe6-4b7f-47b8-bc98-e2ff68572c1b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:40.111445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1039/2070] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:53:39 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:40.115913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675e3e0-359e-46f9-a173-ca8b59e7ad38 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:40.250562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675e3e0-359e-46f9-a173-ca8b59e7ad38 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:40.353474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380cdd7-819b-459f-81f6-f3497ce86376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:40.354217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380cdd7-819b-459f-81f6-f3497ce86376 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:40.354558 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380cdd7-819b-459f-81f6-f3497ce86376 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:40.354839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4380cdd7-819b-459f-81f6-f3497ce86376 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:40.355644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380cdd7-819b-459f-81f6-f3497ce86376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:40.357739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4380cdd7-819b-459f-81f6-f3497ce86376 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:40.400816 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1032/2071] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:53:40 2022] POST /identity/v3/users => generated 295 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:40.401886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0569fe6-4b7f-47b8-bc98-e2ff68572c1b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:40.420957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0569fe6-4b7f-47b8-bc98-e2ff68572c1b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:40.433063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ddb5a-9e8b-4cfb-b74f-326baf859669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:40.434004 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ddb5a-9e8b-4cfb-b74f-326baf859669 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:40.434148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ddb5a-9e8b-4cfb-b74f-326baf859669 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:40.434383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ddb5a-9e8b-4cfb-b74f-326baf859669 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:40.436643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ddb5a-9e8b-4cfb-b74f-326baf859669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:40.436821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ddb5a-9e8b-4cfb-b74f-326baf859669 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:40.439967 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:40.447448 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380cdd7-819b-459f-81f6-f3497ce86376 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:40.449589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4380cdd7-819b-459f-81f6-f3497ce86376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:40.456358 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1040/2072] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:53:40.460919 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ddb5a-9e8b-4cfb-b74f-326baf859669 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:40.472251 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ddb5a-9e8b-4cfb-b74f-326baf859669 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:40.479530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4380cdd7-819b-459f-81f6-f3497ce86376 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:40.485923 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77423fb1-9720-4f61-8fcc-8a2d6813f87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:40.487281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77423fb1-9720-4f61-8fcc-8a2d6813f87c admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:40.488006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77423fb1-9720-4f61-8fcc-8a2d6813f87c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:40.488450 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77423fb1-9720-4f61-8fcc-8a2d6813f87c admin admin] PATH_INFO: `/v3/projects/85325e5849ae45fbb2983d4dbc162dd2/users/c6a012aa944246ba85e4bba13ed18343/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:40.492246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:40.494287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:40.494875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:40.495305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35 admin admin] PATH_INFO: `/v3/projects/0e5ff4cff4fd48c6a1964d76e974ba0a` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:40.496724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e5ff4cff4fd48c6a1964d76e974ba0a)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:40.499358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:40.510202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77423fb1-9720-4f61-8fcc-8a2d6813f87c admin admin] RBAC: Authorizing `identity:create_grant(project_id=85325e5849ae45fbb2983d4dbc162dd2, user_id=c6a012aa944246ba85e4bba13ed18343, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:40.513293 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77423fb1-9720-4f61-8fcc-8a2d6813f87c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:40.529847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1041/2073] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:40 2022] PUT /identity/v3/projects/85325e5849ae45fbb2983d4dbc162dd2/users/c6a012aa944246ba85e4bba13ed18343/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:40.534641 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77423fb1-9720-4f61-8fcc-8a2d6813f87c admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:40.555615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:40.580458 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e5ff4cff4fd48c6a1964d76e974ba0a', 'request_id': 'req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:40.589914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e5ff4cff4fd48c6a1964d76e974ba0a', 'request_id': 'req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:40.597243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e5ff4cff4fd48c6a1964d76e974ba0a', 'request_id': 'req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:40.610323 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35 admin admin] The token cache is being invalidate because project 0e5ff4cff4fd48c6a1964d76e974ba0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:53:40.610614 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:40.611565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-8223fbfb-e4b6-4e08-abbf-c62f06a1fd35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:40.613617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1033/2074] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:53:40 2022] DELETE /identity/v3/projects/0e5ff4cff4fd48c6a1964d76e974ba0a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:40.619583 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b0e72-c93d-4e15-9583-e71cb230fb5b None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:40.620024 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b0e72-c93d-4e15-9583-e71cb230fb5b None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:40.620436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b0e72-c93d-4e15-9583-e71cb230fb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:40.621482 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77423fb1-9720-4f61-8fcc-8a2d6813f87c admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:40.640242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-ff2b0e72-c93d-4e15-9583-e71cb230fb5b None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:40.641572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ff2b0e72-c93d-4e15-9583-e71cb230fb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:40.688018 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f06e47-6635-4930-ae8b-89282fcc5a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:40.688729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f06e47-6635-4930-ae8b-89282fcc5a04 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:40.689022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f06e47-6635-4930-ae8b-89282fcc5a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:40.689304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f06e47-6635-4930-ae8b-89282fcc5a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:40.690043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f06e47-6635-4930-ae8b-89282fcc5a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:40.691651 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f06e47-6635-4930-ae8b-89282fcc5a04 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:40.703440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1042/2075] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:53:40.708911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f06e47-6635-4930-ae8b-89282fcc5a04 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:40.710701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-72f06e47-6635-4930-ae8b-89282fcc5a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:40.716447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1034/2076] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:53:40.724318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7c250-bd94-4003-95ae-6c76f243d524 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:40.724615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7c250-bd94-4003-95ae-6c76f243d524 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:40.724903 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7c250-bd94-4003-95ae-6c76f243d524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:40.728597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f06e47-6635-4930-ae8b-89282fcc5a04 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:40.734202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c4b0fb-852e-49ec-bc73-6c93e04ca0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:40.735232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4b0fb-852e-49ec-bc73-6c93e04ca0b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:40.735446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4b0fb-852e-49ec-bc73-6c93e04ca0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:40.735734 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c4b0fb-852e-49ec-bc73-6c93e04ca0b3 admin admin] PATH_INFO: `/v3/projects/85325e5849ae45fbb2983d4dbc162dd2/users/c6a012aa944246ba85e4bba13ed18343/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:40.738050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4b0fb-852e-49ec-bc73-6c93e04ca0b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85325e5849ae45fbb2983d4dbc162dd2, user_id=c6a012aa944246ba85e4bba13ed18343, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:40.741038 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c4b0fb-852e-49ec-bc73-6c93e04ca0b3 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:40.745403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-01e7c250-bd94-4003-95ae-6c76f243d524 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:40.746685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-01e7c250-bd94-4003-95ae-6c76f243d524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:40.753438 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1043/2077] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:40 2022] PUT /identity/v3/projects/85325e5849ae45fbb2983d4dbc162dd2/users/c6a012aa944246ba85e4bba13ed18343/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:40.763780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e60ee-1f44-4fd9-984a-0ceb0f303ba7 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:40.764141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e60ee-1f44-4fd9-984a-0ceb0f303ba7 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:40.764433 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32e60ee-1f44-4fd9-984a-0ceb0f303ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:40.792226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-c32e60ee-1f44-4fd9-984a-0ceb0f303ba7 None None] MFA Rules not processed for user `c6a012aa944246ba85e4bba13ed18343`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:40.793542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c32e60ee-1f44-4fd9-984a-0ceb0f303ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:40.810755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1035/2078] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:53:40.814744 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7c250-bd94-4003-95ae-6c76f243d524 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:40.823915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7c250-bd94-4003-95ae-6c76f243d524 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:40.829189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf0474-4ee7-4494-880e-39a03e9db479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:40.829941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf0474-4ee7-4494-880e-39a03e9db479 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:40.830870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf0474-4ee7-4494-880e-39a03e9db479 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:40.830870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf0474-4ee7-4494-880e-39a03e9db479 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:40.831152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf0474-4ee7-4494-880e-39a03e9db479 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:53:40.831662 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf0474-4ee7-4494-880e-39a03e9db479 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:40.833856 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf0474-4ee7-4494-880e-39a03e9db479 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:40.849212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1036/2079] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:53:40.853351 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf0474-4ee7-4494-880e-39a03e9db479 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:40.854316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:40.864306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf0474-4ee7-4494-880e-39a03e9db479 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:40.869016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4fd95-8331-4af7-893f-f779d76a44de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:40.869685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4fd95-8331-4af7-893f-f779d76a44de admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:40.869973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4fd95-8331-4af7-893f-f779d76a44de admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:40.870258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df4fd95-8331-4af7-893f-f779d76a44de admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:40.870956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4fd95-8331-4af7-893f-f779d76a44de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:40.872959 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df4fd95-8331-4af7-893f-f779d76a44de admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:40.883254 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1044/2080] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:40 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:53:40.892896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e60ee-1f44-4fd9-984a-0ceb0f303ba7 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:40.894421 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c32e60ee-1f44-4fd9-984a-0ceb0f303ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:40.902363 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1037/2081] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:53:40 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:53:40.908702 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4fd95-8331-4af7-893f-f779d76a44de admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:40.948646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:40.963560 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32e60ee-1f44-4fd9-984a-0ceb0f303ba7 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:40.993835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df4fd95-8331-4af7-893f-f779d76a44de admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:40.998510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941b89a-5de0-4053-ba0b-8dd78745e868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:40.999185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941b89a-5de0-4053-ba0b-8dd78745e868 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:40.999498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941b89a-5de0-4053-ba0b-8dd78745e868 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:40.999783 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b941b89a-5de0-4053-ba0b-8dd78745e868 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:41.000512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941b89a-5de0-4053-ba0b-8dd78745e868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:41.002426 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b941b89a-5de0-4053-ba0b-8dd78745e868 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:41.022539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1038/2082] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:53:40 2022] POST /identity/v3/users => generated 287 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:53:41.026324 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941b89a-5de0-4053-ba0b-8dd78745e868 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:41.032095 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e7f20-5a86-42fd-800e-9ebfb71bc14d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:41.032776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e7f20-5a86-42fd-800e-9ebfb71bc14d service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:41.033097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e7f20-5a86-42fd-800e-9ebfb71bc14d service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:41.033409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4e7f20-5a86-42fd-800e-9ebfb71bc14d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:41.036492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b941b89a-5de0-4053-ba0b-8dd78745e868 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:41.041157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f030820-bbfa-446d-98b0-c43ebebff81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:41.041806 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f030820-bbfa-446d-98b0-c43ebebff81c admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:41.042091 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f030820-bbfa-446d-98b0-c43ebebff81c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:41.042364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f030820-bbfa-446d-98b0-c43ebebff81c admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:41.043088 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f030820-bbfa-446d-98b0-c43ebebff81c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:41.044873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f030820-bbfa-446d-98b0-c43ebebff81c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:41.059006 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1039/2083] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:53:41.060068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f030820-bbfa-446d-98b0-c43ebebff81c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:41.069982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f030820-bbfa-446d-98b0-c43ebebff81c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:41.075061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753b547-61cc-45ad-b371-2c2a91e81238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:41.075842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b547-61cc-45ad-b371-2c2a91e81238 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:41.076251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b547-61cc-45ad-b371-2c2a91e81238 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:41.076562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8753b547-61cc-45ad-b371-2c2a91e81238 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:41.077485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753b547-61cc-45ad-b371-2c2a91e81238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:41.079248 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8753b547-61cc-45ad-b371-2c2a91e81238 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:41.093789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1040/2084] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:53:41.096901 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753b547-61cc-45ad-b371-2c2a91e81238 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:41.097850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e7f20-5a86-42fd-800e-9ebfb71bc14d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:41.099259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4e7f20-5a86-42fd-800e-9ebfb71bc14d service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:41.107494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8753b547-61cc-45ad-b371-2c2a91e81238 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:41.109707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1045/2085] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:40 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:53:41.113811 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eec20a-ec67-4625-80f3-51237caefd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:41.114940 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eec20a-ec67-4625-80f3-51237caefd83 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:41.115344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eec20a-ec67-4625-80f3-51237caefd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:41.115704 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eec20a-ec67-4625-80f3-51237caefd83 admin admin] PATH_INFO: `/v3/projects/230223aca78741fd81f406bdfcd09fe7/users/f7cc4009257d4a38bcce57f12f9d5265/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:41.134881 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eec20a-ec67-4625-80f3-51237caefd83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=230223aca78741fd81f406bdfcd09fe7, user_id=f7cc4009257d4a38bcce57f12f9d5265, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:41.137877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eec20a-ec67-4625-80f3-51237caefd83 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:41.150047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e7f20-5a86-42fd-800e-9ebfb71bc14d service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:41.156451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1041/2086] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:41 2022] PUT /identity/v3/projects/230223aca78741fd81f406bdfcd09fe7/users/f7cc4009257d4a38bcce57f12f9d5265/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:41.159362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4e7f20-5a86-42fd-800e-9ebfb71bc14d service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:41.161259 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01719ca-7227-450d-b7bd-c5fac101ec0b None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:41.161554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01719ca-7227-450d-b7bd-c5fac101ec0b None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:41.161833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01719ca-7227-450d-b7bd-c5fac101ec0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:41.178421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-d01719ca-7227-450d-b7bd-c5fac101ec0b None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:41.179626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d01719ca-7227-450d-b7bd-c5fac101ec0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:41.213122 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5362ae-39f6-4611-9174-888a7b5ef779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:41.213702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5362ae-39f6-4611-9174-888a7b5ef779 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:41.213986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5362ae-39f6-4611-9174-888a7b5ef779 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:41.214267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5362ae-39f6-4611-9174-888a7b5ef779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:41.216133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ca5362ae-39f6-4611-9174-888a7b5ef779 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:41.225726 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:41.240550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1042/2087] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:53:41.250093 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01719ca-7227-450d-b7bd-c5fac101ec0b None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:41.251697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d01719ca-7227-450d-b7bd-c5fac101ec0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:41.279133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5362ae-39f6-4611-9174-888a7b5ef779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:41.280625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5362ae-39f6-4611-9174-888a7b5ef779 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:41.290541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1046/2088] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:53:41.309050 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01719ca-7227-450d-b7bd-c5fac101ec0b None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:41.312738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dedda0-9ee1-483f-973f-42a907ba36cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:41.313515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dedda0-9ee1-483f-973f-42a907ba36cb service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:41.313799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dedda0-9ee1-483f-973f-42a907ba36cb service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:41.314074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dedda0-9ee1-483f-973f-42a907ba36cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:41.323976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dedda0-9ee1-483f-973f-42a907ba36cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:41.325407 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dedda0-9ee1-483f-973f-42a907ba36cb service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:41.336738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1043/2089] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:53:42.136955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec1845-c187-493a-b4a3-c7707faab64e None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:42.137483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec1845-c187-493a-b4a3-c7707faab64e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:42.137577 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec1845-c187-493a-b4a3-c7707faab64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:42.174707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b4ec1845-c187-493a-b4a3-c7707faab64e None None] MFA Rules not processed for user `f7cc4009257d4a38bcce57f12f9d5265`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:42.176100 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b4ec1845-c187-493a-b4a3-c7707faab64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:42.260525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1047/2090] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:42.261857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dedda0-9ee1-483f-973f-42a907ba36cb service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:42.272230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dedda0-9ee1-483f-973f-42a907ba36cb service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:42.277124 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af6a03-7335-4622-840c-e454ae9c1194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:42.277811 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af6a03-7335-4622-840c-e454ae9c1194 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:42.278090 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af6a03-7335-4622-840c-e454ae9c1194 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:42.278360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af6a03-7335-4622-840c-e454ae9c1194 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:42.279128 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af6a03-7335-4622-840c-e454ae9c1194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:42.281568 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af6a03-7335-4622-840c-e454ae9c1194 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:42.295239 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:42.312011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1044/2091] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:53:42 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:42.314404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec1845-c187-493a-b4a3-c7707faab64e None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:42.326762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec1845-c187-493a-b4a3-c7707faab64e None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:42.387611 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3275940-9fce-4be7-b2de-ac08f666795b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:42.388418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3275940-9fce-4be7-b2de-ac08f666795b admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:42.388639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3275940-9fce-4be7-b2de-ac08f666795b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:42.388917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3275940-9fce-4be7-b2de-ac08f666795b admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:42.389617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3275940-9fce-4be7-b2de-ac08f666795b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:42.392216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3275940-9fce-4be7-b2de-ac08f666795b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:42.412599 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1048/2092] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:53:42 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:42.416454 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af6a03-7335-4622-840c-e454ae9c1194 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:42.427195 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af6a03-7335-4622-840c-e454ae9c1194 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:42.431950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67440be6-9860-4ffb-a71d-c312fe6f4c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:42.432619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67440be6-9860-4ffb-a71d-c312fe6f4c4a admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:42.432908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67440be6-9860-4ffb-a71d-c312fe6f4c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:42.433188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67440be6-9860-4ffb-a71d-c312fe6f4c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:42.433947 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67440be6-9860-4ffb-a71d-c312fe6f4c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:42.435563 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67440be6-9860-4ffb-a71d-c312fe6f4c4a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:42.448262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1045/2093] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:53:42.452501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3275940-9fce-4be7-b2de-ac08f666795b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:42.463005 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3275940-9fce-4be7-b2de-ac08f666795b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:42.467597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33b7e7-5a3b-4435-a510-958cf9c90aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:42.468537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33b7e7-5a3b-4435-a510-958cf9c90aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:42.468819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33b7e7-5a3b-4435-a510-958cf9c90aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:42.469097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa33b7e7-5a3b-4435-a510-958cf9c90aa7 admin admin] PATH_INFO: `/v3/projects/000494f9d30e46978b4595905221a272/users/3f51ffe8c8df4e65ac3068d9c6ce9a0a/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:42.484850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33b7e7-5a3b-4435-a510-958cf9c90aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=000494f9d30e46978b4595905221a272, user_id=3f51ffe8c8df4e65ac3068d9c6ce9a0a, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:42.487511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa33b7e7-5a3b-4435-a510-958cf9c90aa7 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:42.501176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1049/2094] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:42 2022] PUT /identity/v3/projects/000494f9d30e46978b4595905221a272/users/3f51ffe8c8df4e65ac3068d9c6ce9a0a/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:42.505215 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67440be6-9860-4ffb-a71d-c312fe6f4c4a admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:42.515859 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67440be6-9860-4ffb-a71d-c312fe6f4c4a admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:42.572228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07714623-39fb-4064-97e0-3142ce68d6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:42.572962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07714623-39fb-4064-97e0-3142ce68d6fc admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:42.573247 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07714623-39fb-4064-97e0-3142ce68d6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:42.573524 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07714623-39fb-4064-97e0-3142ce68d6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:42.574249 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07714623-39fb-4064-97e0-3142ce68d6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:42.575831 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07714623-39fb-4064-97e0-3142ce68d6fc admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:42.586153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1046/2095] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:53:42.590573 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33b7e7-5a3b-4435-a510-958cf9c90aa7 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:42.597342 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:42.601162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa33b7e7-5a3b-4435-a510-958cf9c90aa7 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:42.606635 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf0334-d3b1-46bc-8b64-ee8b17c0567f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:42.607510 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf0334-d3b1-46bc-8b64-ee8b17c0567f admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:42.607786 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf0334-d3b1-46bc-8b64-ee8b17c0567f admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:42.608091 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bf0334-d3b1-46bc-8b64-ee8b17c0567f admin admin] PATH_INFO: `/v3/projects/000494f9d30e46978b4595905221a272/users/3f51ffe8c8df4e65ac3068d9c6ce9a0a/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:42.610038 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf0334-d3b1-46bc-8b64-ee8b17c0567f admin admin] RBAC: Authorizing `identity:create_grant(project_id=000494f9d30e46978b4595905221a272, user_id=3f51ffe8c8df4e65ac3068d9c6ce9a0a, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:42.612691 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bf0334-d3b1-46bc-8b64-ee8b17c0567f admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:42.622732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1050/2096] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:42 2022] PUT /identity/v3/projects/000494f9d30e46978b4595905221a272/users/3f51ffe8c8df4e65ac3068d9c6ce9a0a/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:53:43.442673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718011a5-df8e-4348-a2f2-af009cae33f0 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:43.443390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718011a5-df8e-4348-a2f2-af009cae33f0 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:43.443586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718011a5-df8e-4348-a2f2-af009cae33f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:43.472303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-718011a5-df8e-4348-a2f2-af009cae33f0 None None] MFA Rules not processed for user `3f51ffe8c8df4e65ac3068d9c6ce9a0a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:43.474284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-718011a5-df8e-4348-a2f2-af009cae33f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:43.561397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1047/2097] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:43 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:53:43.572854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf0334-d3b1-46bc-8b64-ee8b17c0567f admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:43.632073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bf0334-d3b1-46bc-8b64-ee8b17c0567f admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:43.635791 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d3a15-49f8-429e-b536-3fa997979cbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:43.636556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d3a15-49f8-429e-b536-3fa997979cbf service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:43.636930 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d3a15-49f8-429e-b536-3fa997979cbf service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:43.637216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d3a15-49f8-429e-b536-3fa997979cbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:43.696497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d3a15-49f8-429e-b536-3fa997979cbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:43.697957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d3a15-49f8-429e-b536-3fa997979cbf service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:43.708047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1051/2098] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:53:46.557446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718011a5-df8e-4348-a2f2-af009cae33f0 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:46.567729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718011a5-df8e-4348-a2f2-af009cae33f0 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:46.571604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e175d5a1-2d34-46ea-b41a-0a544a6f7c0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:46.572283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175d5a1-2d34-46ea-b41a-0a544a6f7c0e service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:46.572572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175d5a1-2d34-46ea-b41a-0a544a6f7c0e service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:46.572857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175d5a1-2d34-46ea-b41a-0a544a6f7c0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:46.581848 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175d5a1-2d34-46ea-b41a-0a544a6f7c0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:46.583273 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e175d5a1-2d34-46ea-b41a-0a544a6f7c0e service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:46.592648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1048/2099] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:53:53.496818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d3a15-49f8-429e-b536-3fa997979cbf service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:53.573963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d3a15-49f8-429e-b536-3fa997979cbf service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:53.578680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec246a08-94e4-43b6-9f13-d8b1596d03ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:53.579650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec246a08-94e4-43b6-9f13-d8b1596d03ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:53.579971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec246a08-94e4-43b6-9f13-d8b1596d03ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:53.580255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec246a08-94e4-43b6-9f13-d8b1596d03ac admin admin] PATH_INFO: `/v3/users/5b2bdb85b2144a04bbc5aeb4998a09e4` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:53.581456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec246a08-94e4-43b6-9f13-d8b1596d03ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b2bdb85b2144a04bbc5aeb4998a09e4)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:53.583378 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec246a08-94e4-43b6-9f13-d8b1596d03ac admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:53.631588 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ec246a08-94e4-43b6-9f13-d8b1596d03ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b2bdb85b2144a04bbc5aeb4998a09e4'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:53.632749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:53.646828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ec246a08-94e4-43b6-9f13-d8b1596d03ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b2bdb85b2144a04bbc5aeb4998a09e4'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:53.654205 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ec246a08-94e4-43b6-9f13-d8b1596d03ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b2bdb85b2144a04bbc5aeb4998a09e4'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:53.654934 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ec246a08-94e4-43b6-9f13-d8b1596d03ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b2bdb85b2144a04bbc5aeb4998a09e4'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:53.663658 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ec246a08-94e4-43b6-9f13-d8b1596d03ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b2bdb85b2144a04bbc5aeb4998a09e4'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:53.669593 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1052/2100] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:53:53 2022] DELETE /identity/v3/users/5b2bdb85b2144a04bbc5aeb4998a09e4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:53.674614 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790ef9c-e2f1-4e8a-8096-82a0ca42c346 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:53.675085 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790ef9c-e2f1-4e8a-8096-82a0ca42c346 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:53.675180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4790ef9c-e2f1-4e8a-8096-82a0ca42c346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:53.692923 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4790ef9c-e2f1-4e8a-8096-82a0ca42c346 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:53.694141 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4790ef9c-e2f1-4e8a-8096-82a0ca42c346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:53.743408 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:53.760115 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1049/2101] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:53.770321 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec246a08-94e4-43b6-9f13-d8b1596d03ac admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:53.771720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ec246a08-94e4-43b6-9f13-d8b1596d03ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:53.836415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec246a08-94e4-43b6-9f13-d8b1596d03ac admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:53.840726 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4caf9-7c92-4590-90a4-41e18e016a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:53.841385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4caf9-7c92-4590-90a4-41e18e016a3e service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:53.841676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4caf9-7c92-4590-90a4-41e18e016a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:53.841959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e4caf9-7c92-4590-90a4-41e18e016a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:53.852072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4caf9-7c92-4590-90a4-41e18e016a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:53.853499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e4caf9-7c92-4590-90a4-41e18e016a3e service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:53.863753 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1053/2102] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:53:54.136111 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790ef9c-e2f1-4e8a-8096-82a0ca42c346 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:54.137771 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4790ef9c-e2f1-4e8a-8096-82a0ca42c346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:54.151061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4790ef9c-e2f1-4e8a-8096-82a0ca42c346 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:54.157750 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:54.158642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:54.158939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:54.159184 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] PATH_INFO: `/v3/projects/94b7f5f6eae14ba8b3854ed4d0fe24d7` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:54.161551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94b7f5f6eae14ba8b3854ed4d0fe24d7)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:54.164397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:54.224117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94b7f5f6eae14ba8b3854ed4d0fe24d7', 'request_id': 'req-4da1dad7-6122-4834-9d40-4514f2d30ca7'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.231422 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94b7f5f6eae14ba8b3854ed4d0fe24d7', 'request_id': 'req-4da1dad7-6122-4834-9d40-4514f2d30ca7'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.237104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94b7f5f6eae14ba8b3854ed4d0fe24d7', 'request_id': 'req-4da1dad7-6122-4834-9d40-4514f2d30ca7'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.248111 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] The token cache is being invalidate because project 94b7f5f6eae14ba8b3854ed4d0fe24d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:53:54.248452 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.248962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.250997 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1050/2103] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:53:54 2022] DELETE /identity/v3/projects/94b7f5f6eae14ba8b3854ed4d0fe24d7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:54.254961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4caf9-7c92-4590-90a4-41e18e016a3e service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:54.264890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e4caf9-7c92-4590-90a4-41e18e016a3e service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:54.320255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfc61c-2514-4426-a1ab-943120d6efcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:54.321125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfc61c-2514-4426-a1ab-943120d6efcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:54.321409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfc61c-2514-4426-a1ab-943120d6efcb admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:54.321686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfc61c-2514-4426-a1ab-943120d6efcb admin admin] PATH_INFO: `/v3/users/a6715f7f5878486c92f292f686f2b17f` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:54.322858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfc61c-2514-4426-a1ab-943120d6efcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6715f7f5878486c92f292f686f2b17f)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:54.324799 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bfc61c-2514-4426-a1ab-943120d6efcb admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:54.373482 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-85bfc61c-2514-4426-a1ab-943120d6efcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6715f7f5878486c92f292f686f2b17f'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.385598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-85bfc61c-2514-4426-a1ab-943120d6efcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6715f7f5878486c92f292f686f2b17f'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.389265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:54.394263 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-85bfc61c-2514-4426-a1ab-943120d6efcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6715f7f5878486c92f292f686f2b17f'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.395353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-85bfc61c-2514-4426-a1ab-943120d6efcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6715f7f5878486c92f292f686f2b17f'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.402885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-85bfc61c-2514-4426-a1ab-943120d6efcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6715f7f5878486c92f292f686f2b17f'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.408857 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1054/2104] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:53:54 2022] DELETE /identity/v3/users/a6715f7f5878486c92f292f686f2b17f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:54.689612 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:54.691359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:54.707877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1dad7-6122-4834-9d40-4514f2d30ca7 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:54.774392 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:54.775186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:54.775469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:54.775744 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] PATH_INFO: `/v3/projects/14f0a4078b83478890ab8192f7cecaea` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:54.777058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f0a4078b83478890ab8192f7cecaea)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:54.779174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:54.832138 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f0a4078b83478890ab8192f7cecaea', 'request_id': 'req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.842498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f0a4078b83478890ab8192f7cecaea', 'request_id': 'req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.848628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f0a4078b83478890ab8192f7cecaea', 'request_id': 'req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.861108 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] The token cache is being invalidate because project 14f0a4078b83478890ab8192f7cecaea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:53:54.861657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.862584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:54.866069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1051/2105] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:53:54 2022] DELETE /identity/v3/projects/14f0a4078b83478890ab8192f7cecaea => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:54.871224 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfc61c-2514-4426-a1ab-943120d6efcb admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:54.872877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-85bfc61c-2514-4426-a1ab-943120d6efcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:54.892810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfc61c-2514-4426-a1ab-943120d6efcb admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:54.962129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97ed6e-c3c9-4557-a90b-7364381d8ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:54.963128 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97ed6e-c3c9-4557-a90b-7364381d8ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:54.963465 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97ed6e-c3c9-4557-a90b-7364381d8ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:54.963715 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97ed6e-c3c9-4557-a90b-7364381d8ea9 admin admin] PATH_INFO: `/v3/users/c6a012aa944246ba85e4bba13ed18343` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:54.965069 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97ed6e-c3c9-4557-a90b-7364381d8ea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6a012aa944246ba85e4bba13ed18343)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:54.967120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad97ed6e-c3c9-4557-a90b-7364381d8ea9 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:55.022007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ad97ed6e-c3c9-4557-a90b-7364381d8ea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6a012aa944246ba85e4bba13ed18343'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:55.034126 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ad97ed6e-c3c9-4557-a90b-7364381d8ea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6a012aa944246ba85e4bba13ed18343'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:55.042278 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ad97ed6e-c3c9-4557-a90b-7364381d8ea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6a012aa944246ba85e4bba13ed18343'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:55.043017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ad97ed6e-c3c9-4557-a90b-7364381d8ea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6a012aa944246ba85e4bba13ed18343'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:55.051772 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ad97ed6e-c3c9-4557-a90b-7364381d8ea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6a012aa944246ba85e4bba13ed18343'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:55.058299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1055/2106] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:53:54 2022] DELETE /identity/v3/users/c6a012aa944246ba85e4bba13ed18343 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:55.332269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:55.333697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:55.342851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:55.347565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e0002-e4e9-4083-bd56-b2336bdfa67c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:55.411778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8ba2c-4b29-40c5-93d7-4299508279e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:55.412709 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8ba2c-4b29-40c5-93d7-4299508279e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:55.412995 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8ba2c-4b29-40c5-93d7-4299508279e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:55.413739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8ba2c-4b29-40c5-93d7-4299508279e9 admin admin] PATH_INFO: `/v3/projects/85325e5849ae45fbb2983d4dbc162dd2` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:55.414495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8ba2c-4b29-40c5-93d7-4299508279e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85325e5849ae45fbb2983d4dbc162dd2)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:55.416582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8ba2c-4b29-40c5-93d7-4299508279e9 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:55.463734 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-09b8ba2c-4b29-40c5-93d7-4299508279e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85325e5849ae45fbb2983d4dbc162dd2', 'request_id': 'req-09b8ba2c-4b29-40c5-93d7-4299508279e9'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:55.473120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-09b8ba2c-4b29-40c5-93d7-4299508279e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85325e5849ae45fbb2983d4dbc162dd2', 'request_id': 'req-09b8ba2c-4b29-40c5-93d7-4299508279e9'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:55.479387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-09b8ba2c-4b29-40c5-93d7-4299508279e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85325e5849ae45fbb2983d4dbc162dd2', 'request_id': 'req-09b8ba2c-4b29-40c5-93d7-4299508279e9'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:55.496102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-09b8ba2c-4b29-40c5-93d7-4299508279e9 admin admin] The token cache is being invalidate because project 85325e5849ae45fbb2983d4dbc162dd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:53:55.496415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-09b8ba2c-4b29-40c5-93d7-4299508279e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:55.497102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-09b8ba2c-4b29-40c5-93d7-4299508279e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:55.498754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1052/2107] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:53:55 2022] DELETE /identity/v3/projects/85325e5849ae45fbb2983d4dbc162dd2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:55.504711 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281e885-4ed2-4a6f-85d6-e62cff98656f None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:55.505130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281e885-4ed2-4a6f-85d6-e62cff98656f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:55.505424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d281e885-4ed2-4a6f-85d6-e62cff98656f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:55.523626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-d281e885-4ed2-4a6f-85d6-e62cff98656f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:55.524847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d281e885-4ed2-4a6f-85d6-e62cff98656f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:55.598609 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1056/2108] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:55.608213 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c97829-64d5-4b73-8cf6-4c8cce427c26 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:55.609504 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c97829-64d5-4b73-8cf6-4c8cce427c26 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:55.609865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c97829-64d5-4b73-8cf6-4c8cce427c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:55.638856 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-65c97829-64d5-4b73-8cf6-4c8cce427c26 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:55.640135 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-65c97829-64d5-4b73-8cf6-4c8cce427c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:55.652127 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1053/2109] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:55.658216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281e885-4ed2-4a6f-85d6-e62cff98656f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:55.674995 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d281e885-4ed2-4a6f-85d6-e62cff98656f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:55.683298 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e41a5a-4852-48fb-9803-6477da8db7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:55.684028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e41a5a-4852-48fb-9803-6477da8db7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:55.684323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e41a5a-4852-48fb-9803-6477da8db7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:55.684608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e41a5a-4852-48fb-9803-6477da8db7d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:55.685351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e41a5a-4852-48fb-9803-6477da8db7d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:53:55.685661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e41a5a-4852-48fb-9803-6477da8db7d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:55.687302 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e41a5a-4852-48fb-9803-6477da8db7d0 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:55.689133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:55.706051 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1057/2110] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:53:55.707122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c97829-64d5-4b73-8cf6-4c8cce427c26 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:55.717891 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c97829-64d5-4b73-8cf6-4c8cce427c26 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:55.741711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f91f53-00e5-4ded-a325-a93173e78b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:55.742400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f91f53-00e5-4ded-a325-a93173e78b4b admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:55.742691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f91f53-00e5-4ded-a325-a93173e78b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:55.742972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f91f53-00e5-4ded-a325-a93173e78b4b admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:55.743679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f91f53-00e5-4ded-a325-a93173e78b4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:55.745626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f91f53-00e5-4ded-a325-a93173e78b4b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:55.791275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1054/2111] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:53:55 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:55.794353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e41a5a-4852-48fb-9803-6477da8db7d0 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:55.805783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e41a5a-4852-48fb-9803-6477da8db7d0 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:55.895272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf722043-74ac-4513-a279-170f475fe825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:55.896117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf722043-74ac-4513-a279-170f475fe825 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:55.896497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf722043-74ac-4513-a279-170f475fe825 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:55.896881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf722043-74ac-4513-a279-170f475fe825 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:55.897642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf722043-74ac-4513-a279-170f475fe825 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:55.899793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf722043-74ac-4513-a279-170f475fe825 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:55.935371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1058/2112] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:53:55 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:55.940145 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f91f53-00e5-4ded-a325-a93173e78b4b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:55.953581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f91f53-00e5-4ded-a325-a93173e78b4b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:55.959159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c7f44-5791-4bbe-a54d-037a1348f2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:55.959874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c7f44-5791-4bbe-a54d-037a1348f2fa admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:55.960191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c7f44-5791-4bbe-a54d-037a1348f2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:55.960477 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c7f44-5791-4bbe-a54d-037a1348f2fa admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:55.961220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c7f44-5791-4bbe-a54d-037a1348f2fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:55.962935 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c7f44-5791-4bbe-a54d-037a1348f2fa admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:55.983528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1055/2113] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:53:55.988505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf722043-74ac-4513-a279-170f475fe825 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:56.000963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf722043-74ac-4513-a279-170f475fe825 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:56.012457 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b15a3a-4e85-4463-8a24-e4f0d0bd67b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:56.013336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b15a3a-4e85-4463-8a24-e4f0d0bd67b0 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:56.013772 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b15a3a-4e85-4463-8a24-e4f0d0bd67b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:56.014217 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b15a3a-4e85-4463-8a24-e4f0d0bd67b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:56.015102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b15a3a-4e85-4463-8a24-e4f0d0bd67b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:56.017459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b15a3a-4e85-4463-8a24-e4f0d0bd67b0 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:56.042496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1059/2114] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:55 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:53:56.044611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c7f44-5791-4bbe-a54d-037a1348f2fa admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:56.056326 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c7f44-5791-4bbe-a54d-037a1348f2fa admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:56.062581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc71ad4-921d-4620-8bae-1fdfedaa8f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:56.064264 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc71ad4-921d-4620-8bae-1fdfedaa8f09 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:56.064580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc71ad4-921d-4620-8bae-1fdfedaa8f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:56.065069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc71ad4-921d-4620-8bae-1fdfedaa8f09 admin admin] PATH_INFO: `/v3/projects/01603c92954a4777870ca0a95d912fb8/users/5a3dfbe73eb444dc8163bd9cbcb3ae8c/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:56.090820 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc71ad4-921d-4620-8bae-1fdfedaa8f09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01603c92954a4777870ca0a95d912fb8, user_id=5a3dfbe73eb444dc8163bd9cbcb3ae8c, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:56.094777 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc71ad4-921d-4620-8bae-1fdfedaa8f09 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:56.117781 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1056/2115] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:56 2022] PUT /identity/v3/projects/01603c92954a4777870ca0a95d912fb8/users/5a3dfbe73eb444dc8163bd9cbcb3ae8c/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:56.131557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966977ad-7de8-4c2d-9e21-530147e2733a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:56.132299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966977ad-7de8-4c2d-9e21-530147e2733a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:56.132747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966977ad-7de8-4c2d-9e21-530147e2733a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:56.157754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc71ad4-921d-4620-8bae-1fdfedaa8f09 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:56.159792 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-afc71ad4-921d-4620-8bae-1fdfedaa8f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:56.187537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-966977ad-7de8-4c2d-9e21-530147e2733a None None] MFA Rules not processed for user `5a3dfbe73eb444dc8163bd9cbcb3ae8c`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:56.189445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-966977ad-7de8-4c2d-9e21-530147e2733a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:56.210500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:56.253316 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc71ad4-921d-4620-8bae-1fdfedaa8f09 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:56.259355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99275a-24eb-49e6-a675-4f3d75a9d917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:56.260842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99275a-24eb-49e6-a675-4f3d75a9d917 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:56.260842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99275a-24eb-49e6-a675-4f3d75a9d917 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:56.261130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99275a-24eb-49e6-a675-4f3d75a9d917 admin admin] PATH_INFO: `/v3/users/9045f10eb4954c5791e5fce5f69b1d82` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:56.262120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99275a-24eb-49e6-a675-4f3d75a9d917 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9045f10eb4954c5791e5fce5f69b1d82)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:56.264146 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99275a-24eb-49e6-a675-4f3d75a9d917 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:56.276351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1060/2116] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:56.287947 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966977ad-7de8-4c2d-9e21-530147e2733a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:56.289362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-966977ad-7de8-4c2d-9e21-530147e2733a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:56.326033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ac99275a-24eb-49e6-a675-4f3d75a9d917 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9045f10eb4954c5791e5fce5f69b1d82'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:56.339375 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ac99275a-24eb-49e6-a675-4f3d75a9d917 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9045f10eb4954c5791e5fce5f69b1d82'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:56.340166 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ac99275a-24eb-49e6-a675-4f3d75a9d917 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9045f10eb4954c5791e5fce5f69b1d82'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:56.349351 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ac99275a-24eb-49e6-a675-4f3d75a9d917 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9045f10eb4954c5791e5fce5f69b1d82'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:56.350362 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ac99275a-24eb-49e6-a675-4f3d75a9d917 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9045f10eb4954c5791e5fce5f69b1d82'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:56.361524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966977ad-7de8-4c2d-9e21-530147e2733a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:56.365933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ba922-e886-48ff-b568-cfb4eec8b76f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:56.366597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ba922-e886-48ff-b568-cfb4eec8b76f service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:56.366876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ba922-e886-48ff-b568-cfb4eec8b76f service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:56.367157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ba922-e886-48ff-b568-cfb4eec8b76f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:56.367673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1057/2117] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:53:56 2022] DELETE /identity/v3/users/9045f10eb4954c5791e5fce5f69b1d82 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:53:56.372116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d77ba922-e886-48ff-b568-cfb4eec8b76f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:56.372573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25133c-0fb8-492e-8034-735a03c9b8e3 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:56.373178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25133c-0fb8-492e-8034-735a03c9b8e3 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:56.373667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db25133c-0fb8-492e-8034-735a03c9b8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:56.384999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ba922-e886-48ff-b568-cfb4eec8b76f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:56.386461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ba922-e886-48ff-b568-cfb4eec8b76f service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:56.392925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-db25133c-0fb8-492e-8034-735a03c9b8e3 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:56.394133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-db25133c-0fb8-492e-8034-735a03c9b8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:56.395880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1061/2118] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:53:56.415375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ba922-e886-48ff-b568-cfb4eec8b76f service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:56.416820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d77ba922-e886-48ff-b568-cfb4eec8b76f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:56.427735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:56.432808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ba922-e886-48ff-b568-cfb4eec8b76f service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:56.465795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1058/2119] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:53:56.475785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25133c-0fb8-492e-8034-735a03c9b8e3 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:56.477314 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-db25133c-0fb8-492e-8034-735a03c9b8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:56.504524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d5575-dc69-4dbc-9513-bad33d9ee104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:56.505415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d5575-dc69-4dbc-9513-bad33d9ee104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:56.505705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d5575-dc69-4dbc-9513-bad33d9ee104 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:56.505977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d5575-dc69-4dbc-9513-bad33d9ee104 admin admin] PATH_INFO: `/v3/users/5a3dfbe73eb444dc8163bd9cbcb3ae8c` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:56.507697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d5575-dc69-4dbc-9513-bad33d9ee104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a3dfbe73eb444dc8163bd9cbcb3ae8c)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:56.510219 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d5575-dc69-4dbc-9513-bad33d9ee104 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:56.543045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db25133c-0fb8-492e-8034-735a03c9b8e3 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:56.547025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7f4b66-2ef8-418d-8f60-e6790918fd4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:56.548337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f4b66-2ef8-418d-8f60-e6790918fd4a service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:56.548783 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f4b66-2ef8-418d-8f60-e6790918fd4a service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:56.548937 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f4b66-2ef8-418d-8f60-e6790918fd4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:56.556334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:56.562883 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f4b66-2ef8-418d-8f60-e6790918fd4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:56.564588 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7f4b66-2ef8-418d-8f60-e6790918fd4a service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:56.568677 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ff2d5575-dc69-4dbc-9513-bad33d9ee104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a3dfbe73eb444dc8163bd9cbcb3ae8c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:56.578576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1059/2120] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:53:56.587172 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ff2d5575-dc69-4dbc-9513-bad33d9ee104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a3dfbe73eb444dc8163bd9cbcb3ae8c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:56.598404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ff2d5575-dc69-4dbc-9513-bad33d9ee104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a3dfbe73eb444dc8163bd9cbcb3ae8c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:56.599348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ff2d5575-dc69-4dbc-9513-bad33d9ee104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a3dfbe73eb444dc8163bd9cbcb3ae8c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:56.610940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ff2d5575-dc69-4dbc-9513-bad33d9ee104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a3dfbe73eb444dc8163bd9cbcb3ae8c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:56.618474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1062/2121] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:53:56 2022] DELETE /identity/v3/users/5a3dfbe73eb444dc8163bd9cbcb3ae8c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:56.623694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864960b-2cdf-41c5-8d4f-bb20d74a5b53 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:56.624523 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864960b-2cdf-41c5-8d4f-bb20d74a5b53 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:56.625036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6864960b-2cdf-41c5-8d4f-bb20d74a5b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:56.655307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-6864960b-2cdf-41c5-8d4f-bb20d74a5b53 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:56.657372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6864960b-2cdf-41c5-8d4f-bb20d74a5b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:56.776519 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1060/2122] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:56.788338 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d5575-dc69-4dbc-9513-bad33d9ee104 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:56.790210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ff2d5575-dc69-4dbc-9513-bad33d9ee104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:56.815090 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864960b-2cdf-41c5-8d4f-bb20d74a5b53 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:56.816872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6864960b-2cdf-41c5-8d4f-bb20d74a5b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:56.833145 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6864960b-2cdf-41c5-8d4f-bb20d74a5b53 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:56.840626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf0dfd6-8808-457b-b9e5-2ad4757f0af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:56.841780 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0dfd6-8808-457b-b9e5-2ad4757f0af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:56.842101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0dfd6-8808-457b-b9e5-2ad4757f0af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:56.842339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0dfd6-8808-457b-b9e5-2ad4757f0af0 admin admin] PATH_INFO: `/v3/users/dfbd9733da8247f797f5c80d0d2c0ceb` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:56.844958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0dfd6-8808-457b-b9e5-2ad4757f0af0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfbd9733da8247f797f5c80d0d2c0ceb)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:56.847239 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf0dfd6-8808-457b-b9e5-2ad4757f0af0 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:56.871070 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d5575-dc69-4dbc-9513-bad33d9ee104 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:56.876804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3601e8-72f1-4a2f-99e6-04c7905aa7aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:56.877508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3601e8-72f1-4a2f-99e6-04c7905aa7aa service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:56.877846 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3601e8-72f1-4a2f-99e6-04c7905aa7aa service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:56.878074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3601e8-72f1-4a2f-99e6-04c7905aa7aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:56.890055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3601e8-72f1-4a2f-99e6-04c7905aa7aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:56.892552 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3601e8-72f1-4a2f-99e6-04c7905aa7aa service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:56.908187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1063/2123] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:53:56.950713 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-eaf0dfd6-8808-457b-b9e5-2ad4757f0af0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfbd9733da8247f797f5c80d0d2c0ceb'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:56.952128 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3601e8-72f1-4a2f-99e6-04c7905aa7aa service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:56.955096 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-da3601e8-72f1-4a2f-99e6-04c7905aa7aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:56.987457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-eaf0dfd6-8808-457b-b9e5-2ad4757f0af0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfbd9733da8247f797f5c80d0d2c0ceb'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:56.993613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-eaf0dfd6-8808-457b-b9e5-2ad4757f0af0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfbd9733da8247f797f5c80d0d2c0ceb'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:56.996349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3601e8-72f1-4a2f-99e6-04c7905aa7aa service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:57.006733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:57.007719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:57.008055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:57.008375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] PATH_INFO: `/v3/projects/cff6adeadf8f4a80becd517b88133d3c` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:57.011244 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cff6adeadf8f4a80becd517b88133d3c)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:57.011814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-eaf0dfd6-8808-457b-b9e5-2ad4757f0af0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfbd9733da8247f797f5c80d0d2c0ceb'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.013615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-eaf0dfd6-8808-457b-b9e5-2ad4757f0af0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfbd9733da8247f797f5c80d0d2c0ceb'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.019468 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:57.050866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1061/2124] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:53:56 2022] DELETE /identity/v3/users/dfbd9733da8247f797f5c80d0d2c0ceb => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:53:57.052468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b2fce-ae69-43a4-9dfb-d9daeccb090c None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:57.052756 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b2fce-ae69-43a4-9dfb-d9daeccb090c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:57.053042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b2fce-ae69-43a4-9dfb-d9daeccb090c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:57.062291 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:57.086427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-875b2fce-ae69-43a4-9dfb-d9daeccb090c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:57.088460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-875b2fce-ae69-43a4-9dfb-d9daeccb090c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:57.106410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cff6adeadf8f4a80becd517b88133d3c', 'request_id': 'req-217a3b93-7822-4b8f-a84d-24945682ec8e'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.121549 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cff6adeadf8f4a80becd517b88133d3c', 'request_id': 'req-217a3b93-7822-4b8f-a84d-24945682ec8e'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.135369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cff6adeadf8f4a80becd517b88133d3c', 'request_id': 'req-217a3b93-7822-4b8f-a84d-24945682ec8e'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.153499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] The token cache is being invalidate because project cff6adeadf8f4a80becd517b88133d3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:53:57.153986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.154666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.156672 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1064/2125] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:53:56 2022] DELETE /identity/v3/projects/cff6adeadf8f4a80becd517b88133d3c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:57.160658 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:57.162875 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:57.184824 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1062/2126] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:57.195830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b2fce-ae69-43a4-9dfb-d9daeccb090c None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:57.197228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-875b2fce-ae69-43a4-9dfb-d9daeccb090c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:57.211905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:57.288541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a3b93-7822-4b8f-a84d-24945682ec8e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:57.294830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b487dd-a3a7-474e-a194-7a26f3411ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:57.295965 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b487dd-a3a7-474e-a194-7a26f3411ae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:57.296270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b487dd-a3a7-474e-a194-7a26f3411ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:57.296559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b487dd-a3a7-474e-a194-7a26f3411ae3 admin admin] PATH_INFO: `/v3/users/6bb4904174884cb498f816f8fdad6756` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:57.298578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b487dd-a3a7-474e-a194-7a26f3411ae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb4904174884cb498f816f8fdad6756)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:57.303462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b487dd-a3a7-474e-a194-7a26f3411ae3 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:57.316660 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875b2fce-ae69-43a4-9dfb-d9daeccb090c None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:57.322834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc07fb-fec5-44df-9dca-657564451769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:57.324011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc07fb-fec5-44df-9dca-657564451769 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:57.324572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc07fb-fec5-44df-9dca-657564451769 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:57.326235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afc07fb-fec5-44df-9dca-657564451769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:57.339111 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc07fb-fec5-44df-9dca-657564451769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:57.341193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afc07fb-fec5-44df-9dca-657564451769 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:57.360777 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1063/2127] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:53:57.402036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-55b487dd-a3a7-474e-a194-7a26f3411ae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb4904174884cb498f816f8fdad6756'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.421739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc07fb-fec5-44df-9dca-657564451769 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:57.423755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9afc07fb-fec5-44df-9dca-657564451769 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:57.426316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-55b487dd-a3a7-474e-a194-7a26f3411ae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb4904174884cb498f816f8fdad6756'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.438797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-55b487dd-a3a7-474e-a194-7a26f3411ae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb4904174884cb498f816f8fdad6756'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.439670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-55b487dd-a3a7-474e-a194-7a26f3411ae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb4904174884cb498f816f8fdad6756'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.446718 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afc07fb-fec5-44df-9dca-657564451769 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:57.451054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-55b487dd-a3a7-474e-a194-7a26f3411ae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb4904174884cb498f816f8fdad6756'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.455739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:57.457394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:57.457960 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:57.458502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] PATH_INFO: `/v3/projects/01603c92954a4777870ca0a95d912fb8` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:57.460381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=01603c92954a4777870ca0a95d912fb8)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:57.461803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1065/2128] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:53:57 2022] DELETE /identity/v3/users/6bb4904174884cb498f816f8fdad6756 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:53:57.463366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:57.574760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01603c92954a4777870ca0a95d912fb8', 'request_id': 'req-f14a74c4-d805-4d4d-8825-c073f3db1bcc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.584089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01603c92954a4777870ca0a95d912fb8', 'request_id': 'req-f14a74c4-d805-4d4d-8825-c073f3db1bcc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.595626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01603c92954a4777870ca0a95d912fb8', 'request_id': 'req-f14a74c4-d805-4d4d-8825-c073f3db1bcc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.616360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] The token cache is being invalidate because project 01603c92954a4777870ca0a95d912fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:53:57.616783 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.618024 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.624416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1064/2129] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:53:57 2022] DELETE /identity/v3/projects/01603c92954a4777870ca0a95d912fb8 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:53:57.626186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739861-c483-4e60-a6fd-6baaf32673ed None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:57.626820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739861-c483-4e60-a6fd-6baaf32673ed None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:57.627105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4739861-c483-4e60-a6fd-6baaf32673ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:57.658422 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-a4739861-c483-4e60-a6fd-6baaf32673ed None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:57.659689 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a4739861-c483-4e60-a6fd-6baaf32673ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:57.692070 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:57.763289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1066/2130] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:57.771211 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:57.771454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4e47a-f5ad-469c-b592-90874f8ef8d7 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:57.771808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4e47a-f5ad-469c-b592-90874f8ef8d7 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:57.772219 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4e47a-f5ad-469c-b592-90874f8ef8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:57.773698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:57.790010 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a74c4-d805-4d4d-8825-c073f3db1bcc admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:57.796128 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:57.797094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:57.798006 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:57.798356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] PATH_INFO: `/v3/projects/d88e08e98f054dddaa5caec1b0a39db4` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:57.799060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-9db4e47a-f5ad-469c-b592-90874f8ef8d7 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:57.799713 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d88e08e98f054dddaa5caec1b0a39db4)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:57.800390 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9db4e47a-f5ad-469c-b592-90874f8ef8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:57.801812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:57.810721 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1067/2131] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:57.816940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4e47a-f5ad-469c-b592-90874f8ef8d7 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:57.851442 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4e47a-f5ad-469c-b592-90874f8ef8d7 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:57.878540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd88e08e98f054dddaa5caec1b0a39db4', 'request_id': 'req-561a1f14-5251-44c8-ac9b-1853ed5114a0'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.889826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd88e08e98f054dddaa5caec1b0a39db4', 'request_id': 'req-561a1f14-5251-44c8-ac9b-1853ed5114a0'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.897311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd88e08e98f054dddaa5caec1b0a39db4', 'request_id': 'req-561a1f14-5251-44c8-ac9b-1853ed5114a0'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.909277 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] The token cache is being invalidate because project d88e08e98f054dddaa5caec1b0a39db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:53:57.909501 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.910159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:57.911814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1065/2132] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:53:57 2022] DELETE /identity/v3/projects/d88e08e98f054dddaa5caec1b0a39db4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:57.912781 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:57.914158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:57.927326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab21ac-ccbd-421c-9c11-d0193cec5344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:57.928036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab21ac-ccbd-421c-9c11-d0193cec5344 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:57.928330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab21ac-ccbd-421c-9c11-d0193cec5344 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:57.928615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab21ac-ccbd-421c-9c11-d0193cec5344 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:57.929382 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab21ac-ccbd-421c-9c11-d0193cec5344 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:53:57.929695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab21ac-ccbd-421c-9c11-d0193cec5344 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:57.931294 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab21ac-ccbd-421c-9c11-d0193cec5344 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:57.933200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:57.944516 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1068/2133] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:53:57.946138 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa6832-189a-42af-a3c1-70fc9f258a50 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:57.946435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa6832-189a-42af-a3c1-70fc9f258a50 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:57.946720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa6832-189a-42af-a3c1-70fc9f258a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:57.965101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-d2fa6832-189a-42af-a3c1-70fc9f258a50 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:57.966435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d2fa6832-189a-42af-a3c1-70fc9f258a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:57.974638 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1069/2134] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:57.975585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa6832-189a-42af-a3c1-70fc9f258a50 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:57.976817 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d2fa6832-189a-42af-a3c1-70fc9f258a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:57.984468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a1f14-5251-44c8-ac9b-1853ed5114a0 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:57.989221 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa6832-189a-42af-a3c1-70fc9f258a50 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:57.989718 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:57.990594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:57.990893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:57.991164 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] PATH_INFO: `/v3/projects/6fc5f78b843d4c5b82da0930258e1eb0` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:57.992398 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fc5f78b843d4c5b82da0930258e1eb0)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:57.993853 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70eba2b-9d7f-49ef-bd85-2c0a0ea07de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:57.994313 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:57.994497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eba2b-9d7f-49ef-bd85-2c0a0ea07de2 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:57.994779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eba2b-9d7f-49ef-bd85-2c0a0ea07de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:57.995063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70eba2b-9d7f-49ef-bd85-2c0a0ea07de2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:57.995732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70eba2b-9d7f-49ef-bd85-2c0a0ea07de2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:57.997637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70eba2b-9d7f-49ef-bd85-2c0a0ea07de2 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.027701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1070/2135] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:53:57 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:58.027701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97f3b2-bd7f-4821-b31e-51b7fa82ebaf None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.027701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97f3b2-bd7f-4821-b31e-51b7fa82ebaf None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.027701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97f3b2-bd7f-4821-b31e-51b7fa82ebaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.041480 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fc5f78b843d4c5b82da0930258e1eb0', 'request_id': 'req-2ac624e7-69a6-4329-863d-13e33607967e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:58.044585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-eb97f3b2-bd7f-4821-b31e-51b7fa82ebaf None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:58.045804 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-eb97f3b2-bd7f-4821-b31e-51b7fa82ebaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:58.052828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fc5f78b843d4c5b82da0930258e1eb0', 'request_id': 'req-2ac624e7-69a6-4329-863d-13e33607967e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:58.058587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fc5f78b843d4c5b82da0930258e1eb0', 'request_id': 'req-2ac624e7-69a6-4329-863d-13e33607967e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:58.069684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] The token cache is being invalidate because project 6fc5f78b843d4c5b82da0930258e1eb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:53:58.070300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:58.070988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:53:58.072864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1066/2136] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:53:57 2022] DELETE /identity/v3/projects/6fc5f78b843d4c5b82da0930258e1eb0 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:58.073339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.074625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:58.113542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1071/2137] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:58.115394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f12fd8-0ca2-414a-8c19-f30d7c850940 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.115695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f12fd8-0ca2-414a-8c19-f30d7c850940 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.116009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f12fd8-0ca2-414a-8c19-f30d7c850940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.138486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-08f12fd8-0ca2-414a-8c19-f30d7c850940 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:58.139698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-08f12fd8-0ca2-414a-8c19-f30d7c850940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:58.141255 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac624e7-69a6-4329-863d-13e33607967e admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.146014 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19b5d7-a67f-4d7f-b364-f5df41d2f723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.146689 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19b5d7-a67f-4d7f-b364-f5df41d2f723 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.146975 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19b5d7-a67f-4d7f-b364-f5df41d2f723 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.147250 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19b5d7-a67f-4d7f-b364-f5df41d2f723 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.147969 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19b5d7-a67f-4d7f-b364-f5df41d2f723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.148483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1072/2138] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:58.149117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f12fd8-0ca2-414a-8c19-f30d7c850940 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.156025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19b5d7-a67f-4d7f-b364-f5df41d2f723 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.159523 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f12fd8-0ca2-414a-8c19-f30d7c850940 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.164010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45afec7-553d-4056-baaa-be7739f7c987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.164676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45afec7-553d-4056-baaa-be7739f7c987 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.165256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45afec7-553d-4056-baaa-be7739f7c987 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.165548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45afec7-553d-4056-baaa-be7739f7c987 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.166295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45afec7-553d-4056-baaa-be7739f7c987 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:53:58.166611 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45afec7-553d-4056-baaa-be7739f7c987 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.168305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45afec7-553d-4056-baaa-be7739f7c987 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.184174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1067/2139] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:53:58 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:58.184384 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1073/2140] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:53:58.184470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120671d-c8c0-4e02-9997-fd2bf565402d None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.184470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120671d-c8c0-4e02-9997-fd2bf565402d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.184470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4120671d-c8c0-4e02-9997-fd2bf565402d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.185308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45afec7-553d-4056-baaa-be7739f7c987 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.195417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45afec7-553d-4056-baaa-be7739f7c987 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.200073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c0a16-909a-4b5a-8c5b-2ffbbb999b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.200727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c0a16-909a-4b5a-8c5b-2ffbbb999b38 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.201010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c0a16-909a-4b5a-8c5b-2ffbbb999b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.201283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c0a16-909a-4b5a-8c5b-2ffbbb999b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.201840 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4120671d-c8c0-4e02-9997-fd2bf565402d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:58.202000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c0a16-909a-4b5a-8c5b-2ffbbb999b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.203049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4120671d-c8c0-4e02-9997-fd2bf565402d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:58.203598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c0a16-909a-4b5a-8c5b-2ffbbb999b38 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.205803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:58.211213 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1068/2141] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:58.212157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120671d-c8c0-4e02-9997-fd2bf565402d None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.220009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1074/2142] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:53:58.220786 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c0a16-909a-4b5a-8c5b-2ffbbb999b38 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.222429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4120671d-c8c0-4e02-9997-fd2bf565402d None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.227766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ede5bb-b79f-488f-8303-0f5aae8dac02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.228475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ede5bb-b79f-488f-8303-0f5aae8dac02 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.228765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ede5bb-b79f-488f-8303-0f5aae8dac02 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.229043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ede5bb-b79f-488f-8303-0f5aae8dac02 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.229741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ede5bb-b79f-488f-8303-0f5aae8dac02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.230453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c0a16-909a-4b5a-8c5b-2ffbbb999b38 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.231643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ede5bb-b79f-488f-8303-0f5aae8dac02 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.235637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb95cb8-aff6-4310-859f-3bfaed9cdc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.236406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb95cb8-aff6-4310-859f-3bfaed9cdc39 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.236696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb95cb8-aff6-4310-859f-3bfaed9cdc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.236981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb95cb8-aff6-4310-859f-3bfaed9cdc39 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.237707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb95cb8-aff6-4310-859f-3bfaed9cdc39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:53:58.238019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb95cb8-aff6-4310-859f-3bfaed9cdc39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.239598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb95cb8-aff6-4310-859f-3bfaed9cdc39 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.251954 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1075/2143] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:53:58.256220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb95cb8-aff6-4310-859f-3bfaed9cdc39 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.258451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1069/2144] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:53:58 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:58.259248 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ede5bb-b79f-488f-8303-0f5aae8dac02 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.266794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb95cb8-aff6-4310-859f-3bfaed9cdc39 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.269772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:58.325194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5c7d3-0829-4bc8-a0ee-ed05e77aafb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.325868 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5c7d3-0829-4bc8-a0ee-ed05e77aafb2 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.326150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5c7d3-0829-4bc8-a0ee-ed05e77aafb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.326428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db5c7d3-0829-4bc8-a0ee-ed05e77aafb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.327300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5c7d3-0829-4bc8-a0ee-ed05e77aafb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.328443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ede5bb-b79f-488f-8303-0f5aae8dac02 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.329666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db5c7d3-0829-4bc8-a0ee-ed05e77aafb2 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.333569 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68462ab5-35e0-48c5-a2fb-8b93d643c614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.334227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68462ab5-35e0-48c5-a2fb-8b93d643c614 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.334507 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68462ab5-35e0-48c5-a2fb-8b93d643c614 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.334788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68462ab5-35e0-48c5-a2fb-8b93d643c614 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.335531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68462ab5-35e0-48c5-a2fb-8b93d643c614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.337449 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68462ab5-35e0-48c5-a2fb-8b93d643c614 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.343487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1076/2145] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:53:58.343487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5c7d3-0829-4bc8-a0ee-ed05e77aafb2 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.352038 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db5c7d3-0829-4bc8-a0ee-ed05e77aafb2 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.356724 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db8458-057d-42bc-aa74-73a6795c22c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.357473 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db8458-057d-42bc-aa74-73a6795c22c6 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.357761 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db8458-057d-42bc-aa74-73a6795c22c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.358155 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db8458-057d-42bc-aa74-73a6795c22c6 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.358850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db8458-057d-42bc-aa74-73a6795c22c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.360844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db8458-057d-42bc-aa74-73a6795c22c6 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.368567 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1070/2146] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:53:58 2022] POST /identity/v3/projects => generated 391 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:58.369793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68462ab5-35e0-48c5-a2fb-8b93d643c614 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.385710 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1077/2147] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:53:58 2022] POST /identity/v3/users => generated 285 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:58.390359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db8458-057d-42bc-aa74-73a6795c22c6 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.403533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db8458-057d-42bc-aa74-73a6795c22c6 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.462158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68462ab5-35e0-48c5-a2fb-8b93d643c614 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.466192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c793a7-58fd-4c7b-8899-34e1e457675b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.466891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c793a7-58fd-4c7b-8899-34e1e457675b admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.467201 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c793a7-58fd-4c7b-8899-34e1e457675b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.467321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641fc01-894d-4342-9f56-0a7e7033cd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.467454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c793a7-58fd-4c7b-8899-34e1e457675b admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.468233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641fc01-894d-4342-9f56-0a7e7033cd90 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.468384 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c793a7-58fd-4c7b-8899-34e1e457675b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.468619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641fc01-894d-4342-9f56-0a7e7033cd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.468791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d641fc01-894d-4342-9f56-0a7e7033cd90 admin admin] PATH_INFO: `/v3/projects/6c04c69de4f746b98596b3538a77f57b/users/c3e557927b1249e49f793815650f1345/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.470733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c793a7-58fd-4c7b-8899-34e1e457675b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.485444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641fc01-894d-4342-9f56-0a7e7033cd90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c04c69de4f746b98596b3538a77f57b, user_id=c3e557927b1249e49f793815650f1345, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.488159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d641fc01-894d-4342-9f56-0a7e7033cd90 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.495073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1078/2148] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:53:58 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:58.496204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c793a7-58fd-4c7b-8899-34e1e457675b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.507265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c793a7-58fd-4c7b-8899-34e1e457675b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.508250 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1071/2149] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:58 2022] PUT /identity/v3/projects/6c04c69de4f746b98596b3538a77f57b/users/c3e557927b1249e49f793815650f1345/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:58.509034 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641fc01-894d-4342-9f56-0a7e7033cd90 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.577185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:58.591575 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0513a-f315-4be7-a0cd-9d334c40d9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.592327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0513a-f315-4be7-a0cd-9d334c40d9ae admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.592832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0513a-f315-4be7-a0cd-9d334c40d9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.593277 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0513a-f315-4be7-a0cd-9d334c40d9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.594092 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0513a-f315-4be7-a0cd-9d334c40d9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.596413 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0513a-f315-4be7-a0cd-9d334c40d9ae admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.596540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d641fc01-894d-4342-9f56-0a7e7033cd90 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.605455 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6454a-198d-4e0b-bf7f-61c34dcad0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.606344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6454a-198d-4e0b-bf7f-61c34dcad0f3 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.606719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6454a-198d-4e0b-bf7f-61c34dcad0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.607284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6454a-198d-4e0b-bf7f-61c34dcad0f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.608348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6454a-198d-4e0b-bf7f-61c34dcad0f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.608483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1079/2150] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:53:58.609733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cb1df-13c2-4703-82bf-c06f922c8286 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.610112 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cb1df-13c2-4703-82bf-c06f922c8286 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.610262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6454a-198d-4e0b-bf7f-61c34dcad0f3 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.610514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06cb1df-13c2-4703-82bf-c06f922c8286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.634120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1072/2151] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:53:58.634120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6454a-198d-4e0b-bf7f-61c34dcad0f3 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.643258 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-d06cb1df-13c2-4703-82bf-c06f922c8286 None None] MFA Rules not processed for user `c3e557927b1249e49f793815650f1345`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:58.644591 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d06cb1df-13c2-4703-82bf-c06f922c8286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:58.648873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6454a-198d-4e0b-bf7f-61c34dcad0f3 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.656209 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035f270-ccdf-4949-b03d-e9dfe5f4d197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.657744 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035f270-ccdf-4949-b03d-e9dfe5f4d197 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.658200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035f270-ccdf-4949-b03d-e9dfe5f4d197 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.658479 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035f270-ccdf-4949-b03d-e9dfe5f4d197 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.659533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035f270-ccdf-4949-b03d-e9dfe5f4d197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.661279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035f270-ccdf-4949-b03d-e9dfe5f4d197 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.666927 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:58.674734 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1073/2152] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:53:58.675593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035f270-ccdf-4949-b03d-e9dfe5f4d197 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.691995 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035f270-ccdf-4949-b03d-e9dfe5f4d197 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.697812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc9585-b13a-4927-9278-c0fc63d1f693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.698652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc9585-b13a-4927-9278-c0fc63d1f693 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.699133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc9585-b13a-4927-9278-c0fc63d1f693 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.699467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc9585-b13a-4927-9278-c0fc63d1f693 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.708360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc9585-b13a-4927-9278-c0fc63d1f693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.710513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc9585-b13a-4927-9278-c0fc63d1f693 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.726078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1074/2153] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:53:58.726078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc9585-b13a-4927-9278-c0fc63d1f693 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.734111 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc9585-b13a-4927-9278-c0fc63d1f693 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.735333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1080/2154] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:58.736190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06cb1df-13c2-4703-82bf-c06f922c8286 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.742933 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdca827-6997-4cb6-8371-66129adef15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.743871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca827-6997-4cb6-8371-66129adef15b admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.744182 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca827-6997-4cb6-8371-66129adef15b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.744457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdca827-6997-4cb6-8371-66129adef15b admin admin] PATH_INFO: `/v3/projects/f329efa85eee4255b208df73dafb3732/users/2a5727624fa84f1da109dbe438ef750e/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.750066 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06cb1df-13c2-4703-82bf-c06f922c8286 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.755917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1420b-a0a0-4a3f-bd19-6525a9c08799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.756947 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1420b-a0a0-4a3f-bd19-6525a9c08799 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.757326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1420b-a0a0-4a3f-bd19-6525a9c08799 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.757717 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1420b-a0a0-4a3f-bd19-6525a9c08799 admin admin] PATH_INFO: `/v3/projects/0fb227a492654ab4a7f9ff1df08c510f/users/6ffc5057c58a434b812fbeea81a37a2b/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.762675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdca827-6997-4cb6-8371-66129adef15b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f329efa85eee4255b208df73dafb3732, user_id=2a5727624fa84f1da109dbe438ef750e, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.765617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdca827-6997-4cb6-8371-66129adef15b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.776247 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1420b-a0a0-4a3f-bd19-6525a9c08799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb227a492654ab4a7f9ff1df08c510f, user_id=6ffc5057c58a434b812fbeea81a37a2b, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.779010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1420b-a0a0-4a3f-bd19-6525a9c08799 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.782025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1075/2155] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:58 2022] PUT /identity/v3/projects/f329efa85eee4255b208df73dafb3732/users/2a5727624fa84f1da109dbe438ef750e/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:58.782942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdca827-6997-4cb6-8371-66129adef15b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.792189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-2bdca827-6997-4cb6-8371-66129adef15b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:58.799315 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1081/2156] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:58 2022] PUT /identity/v3/projects/0fb227a492654ab4a7f9ff1df08c510f/users/6ffc5057c58a434b812fbeea81a37a2b/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:58.800765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d566a08-ed94-4502-ace0-cf0a029df874 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.801155 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d566a08-ed94-4502-ace0-cf0a029df874 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.801529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d566a08-ed94-4502-ace0-cf0a029df874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.836361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-1d566a08-ed94-4502-ace0-cf0a029df874 None None] MFA Rules not processed for user `2a5727624fa84f1da109dbe438ef750e`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:58.837614 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1d566a08-ed94-4502-ace0-cf0a029df874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:58.854978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdca827-6997-4cb6-8371-66129adef15b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:58.907625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1082/2157] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:58 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:58.909249 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27079108-21ae-455a-8cf8-4e7e4f83ab83 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.909549 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27079108-21ae-455a-8cf8-4e7e4f83ab83 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.909878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27079108-21ae-455a-8cf8-4e7e4f83ab83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.917914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a60856-aca5-4e9f-b083-a90153a9cfdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:58.918707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a60856-aca5-4e9f-b083-a90153a9cfdb service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:58.919096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a60856-aca5-4e9f-b083-a90153a9cfdb service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:58.919518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a60856-aca5-4e9f-b083-a90153a9cfdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:58.942302 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-27079108-21ae-455a-8cf8-4e7e4f83ab83 None None] MFA Rules not processed for user `6ffc5057c58a434b812fbeea81a37a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:58.943528 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-27079108-21ae-455a-8cf8-4e7e4f83ab83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:58.979564 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:58.980887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a60856-aca5-4e9f-b083-a90153a9cfdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:58.982843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a60856-aca5-4e9f-b083-a90153a9cfdb service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:58.992532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1076/2158] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:53:58.993337 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a60856-aca5-4e9f-b083-a90153a9cfdb service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:58.994675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d0a60856-aca5-4e9f-b083-a90153a9cfdb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:59.019315 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1083/2159] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:59.020654 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27079108-21ae-455a-8cf8-4e7e4f83ab83 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:59.048686 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:59.098059 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a60856-aca5-4e9f-b083-a90153a9cfdb service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:59.101968 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd0f0be-1248-4de2-9291-16b7bab30288 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2b4bf2aa1744a46b6d2a1b5a136a0e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:59.102745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0f0be-1248-4de2-9291-16b7bab30288 service glance] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:59.103047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0f0be-1248-4de2-9291-16b7bab30288 service glance] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:59.103327 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd0f0be-1248-4de2-9291-16b7bab30288 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:59.111856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27079108-21ae-455a-8cf8-4e7e4f83ab83 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:59.113776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd0f0be-1248-4de2-9291-16b7bab30288 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:59.115237 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd0f0be-1248-4de2-9291-16b7bab30288 service glance] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:59.120584 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e024f8-cb37-4a6e-87f6-cc2cd7cfc3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:59.122779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e024f8-cb37-4a6e-87f6-cc2cd7cfc3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:59.123917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e024f8-cb37-4a6e-87f6-cc2cd7cfc3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:59.124229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e024f8-cb37-4a6e-87f6-cc2cd7cfc3c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:59.125405 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e024f8-cb37-4a6e-87f6-cc2cd7cfc3c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:59.126673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1077/2160] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:53:59.128641 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e024f8-cb37-4a6e-87f6-cc2cd7cfc3c2 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:59.163855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1084/2161] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:53:59 2022] POST /identity/v3/projects => generated 393 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:53:59.164933 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd0f0be-1248-4de2-9291-16b7bab30288 service glance] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:59.179458 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd0f0be-1248-4de2-9291-16b7bab30288 service glance] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:59.259002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a43a2b-1626-46d5-950a-fe5e18df6967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:59.259813 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a43a2b-1626-46d5-950a-fe5e18df6967 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:59.260137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a43a2b-1626-46d5-950a-fe5e18df6967 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:59.260418 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a43a2b-1626-46d5-950a-fe5e18df6967 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:59.261169 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a43a2b-1626-46d5-950a-fe5e18df6967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:59.263291 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a43a2b-1626-46d5-950a-fe5e18df6967 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:59.285044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1078/2162] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:53:59 2022] POST /identity/v3/users => generated 303 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:53:59.288961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e024f8-cb37-4a6e-87f6-cc2cd7cfc3c2 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:59.302100 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e024f8-cb37-4a6e-87f6-cc2cd7cfc3c2 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:59.307269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4c44d-fa22-4577-8ad7-2832da3424b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:59.316044 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4c44d-fa22-4577-8ad7-2832da3424b6 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:59.316529 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4c44d-fa22-4577-8ad7-2832da3424b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:59.316923 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4c44d-fa22-4577-8ad7-2832da3424b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:59.318320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4c44d-fa22-4577-8ad7-2832da3424b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:59.321523 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4c44d-fa22-4577-8ad7-2832da3424b6 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:59.333554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1085/2163] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:53:59.338057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a43a2b-1626-46d5-950a-fe5e18df6967 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:59.349148 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a43a2b-1626-46d5-950a-fe5e18df6967 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:59.354349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315d584-4504-4ea0-b89c-3308245afa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:59.355286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315d584-4504-4ea0-b89c-3308245afa9e admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:59.355577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315d584-4504-4ea0-b89c-3308245afa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:59.355860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315d584-4504-4ea0-b89c-3308245afa9e admin admin] PATH_INFO: `/v3/projects/a96aa24a3ce640c9b926e3f9a7af4fb7/users/d1dfce23a5e64316bd65d3183ccf5eb2/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:59.374271 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315d584-4504-4ea0-b89c-3308245afa9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a96aa24a3ce640c9b926e3f9a7af4fb7, user_id=d1dfce23a5e64316bd65d3183ccf5eb2, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:59.377664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315d584-4504-4ea0-b89c-3308245afa9e admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:59.394444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1079/2164] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:59 2022] PUT /identity/v3/projects/a96aa24a3ce640c9b926e3f9a7af4fb7/users/d1dfce23a5e64316bd65d3183ccf5eb2/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:53:59.398540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4c44d-fa22-4577-8ad7-2832da3424b6 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:59.409303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4c44d-fa22-4577-8ad7-2832da3424b6 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:59.468754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb39235-8827-4c0f-bdd5-4150a1c96d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:59.469633 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb39235-8827-4c0f-bdd5-4150a1c96d3a admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:59.470014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb39235-8827-4c0f-bdd5-4150a1c96d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:59.470392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb39235-8827-4c0f-bdd5-4150a1c96d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:59.471193 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb39235-8827-4c0f-bdd5-4150a1c96d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:59.473109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb39235-8827-4c0f-bdd5-4150a1c96d3a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:59.486748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1086/2165] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:53:59 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:53:59.492664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315d584-4504-4ea0-b89c-3308245afa9e admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:59.504642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315d584-4504-4ea0-b89c-3308245afa9e admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:59.509556 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c992fa-04a6-462c-891c-470195b1d3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:59.510653 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c992fa-04a6-462c-891c-470195b1d3df admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:59.510942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c992fa-04a6-462c-891c-470195b1d3df admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:59.511219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c992fa-04a6-462c-891c-470195b1d3df admin admin] PATH_INFO: `/v3/projects/a96aa24a3ce640c9b926e3f9a7af4fb7/users/d1dfce23a5e64316bd65d3183ccf5eb2/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:59.513742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c992fa-04a6-462c-891c-470195b1d3df admin admin] RBAC: Authorizing `identity:create_grant(project_id=a96aa24a3ce640c9b926e3f9a7af4fb7, user_id=d1dfce23a5e64316bd65d3183ccf5eb2, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:59.517132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c992fa-04a6-462c-891c-470195b1d3df admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:59.529219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1080/2166] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:53:59 2022] PUT /identity/v3/projects/a96aa24a3ce640c9b926e3f9a7af4fb7/users/d1dfce23a5e64316bd65d3183ccf5eb2/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:53:59.543516 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751ff4a-1665-4479-aa49-785999847b99 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:59.544149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751ff4a-1665-4479-aa49-785999847b99 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:59.544646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751ff4a-1665-4479-aa49-785999847b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:59.575356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-4751ff4a-1665-4479-aa49-785999847b99 None None] MFA Rules not processed for user `d1dfce23a5e64316bd65d3183ccf5eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:53:59.576634 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4751ff4a-1665-4479-aa49-785999847b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:59.613817 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:53:59.661218 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1087/2167] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:53:59 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:53:59.680353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c992fa-04a6-462c-891c-470195b1d3df admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:59.682224 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-87c992fa-04a6-462c-891c-470195b1d3df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:53:59.746267 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c992fa-04a6-462c-891c-470195b1d3df admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:59.750591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc821e-4597-4810-8e77-cb2514800a07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:59.751223 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc821e-4597-4810-8e77-cb2514800a07 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:59.751609 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc821e-4597-4810-8e77-cb2514800a07 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:59.752000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc821e-4597-4810-8e77-cb2514800a07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:59.761644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc821e-4597-4810-8e77-cb2514800a07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:59.763085 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc821e-4597-4810-8e77-cb2514800a07 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:59.772786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1081/2168] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:59 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:53:59.835902 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751ff4a-1665-4479-aa49-785999847b99 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:53:59.850344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4751ff4a-1665-4479-aa49-785999847b99 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:53:59.854408 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27468bef-b9ca-4615-a185-c82616f6e2be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:53:59.855228 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27468bef-b9ca-4615-a185-c82616f6e2be service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:53:59.855613 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27468bef-b9ca-4615-a185-c82616f6e2be service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:53:59.855983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27468bef-b9ca-4615-a185-c82616f6e2be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:53:59.926992 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27468bef-b9ca-4615-a185-c82616f6e2be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:53:59.928896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27468bef-b9ca-4615-a185-c82616f6e2be service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:53:59.940304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1088/2169] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:53:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:54:08.531838 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc821e-4597-4810-8e77-cb2514800a07 service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:08.545542 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:08.598299 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc821e-4597-4810-8e77-cb2514800a07 service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:08.602796 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8e5f6-ec97-4b08-9868-5d7206095b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:08.603678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8e5f6-ec97-4b08-9868-5d7206095b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:08.603993 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8e5f6-ec97-4b08-9868-5d7206095b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:08.604276 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8e5f6-ec97-4b08-9868-5d7206095b3d admin admin] PATH_INFO: `/v3/users/6ffc5057c58a434b812fbeea81a37a2b` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:08.605479 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8e5f6-ec97-4b08-9868-5d7206095b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ffc5057c58a434b812fbeea81a37a2b)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:08.607409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b8e5f6-ec97-4b08-9868-5d7206095b3d admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:08.666112 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-10b8e5f6-ec97-4b08-9868-5d7206095b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ffc5057c58a434b812fbeea81a37a2b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:08.679027 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-10b8e5f6-ec97-4b08-9868-5d7206095b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ffc5057c58a434b812fbeea81a37a2b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:08.679748 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-10b8e5f6-ec97-4b08-9868-5d7206095b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ffc5057c58a434b812fbeea81a37a2b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:08.688049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-10b8e5f6-ec97-4b08-9868-5d7206095b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ffc5057c58a434b812fbeea81a37a2b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:08.689251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-10b8e5f6-ec97-4b08-9868-5d7206095b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ffc5057c58a434b812fbeea81a37a2b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:08.703628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1082/2170] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:54:08 2022] DELETE /identity/v3/users/6ffc5057c58a434b812fbeea81a37a2b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:54:08.708792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8280ac7-d1e6-49b7-b446-1851cb6793d5 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:08.709359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8280ac7-d1e6-49b7-b446-1851cb6793d5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:08.709359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8280ac7-d1e6-49b7-b446-1851cb6793d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:08.729697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-d8280ac7-d1e6-49b7-b446-1851cb6793d5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:08.730994 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d8280ac7-d1e6-49b7-b446-1851cb6793d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:08.799456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1089/2171] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:54:08.808061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8e5f6-ec97-4b08-9868-5d7206095b3d admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:08.809584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-10b8e5f6-ec97-4b08-9868-5d7206095b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:08.886904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8e5f6-ec97-4b08-9868-5d7206095b3d admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:08.891171 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf61e21-0a83-4b38-ab4c-8959c74bdc1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:08.892478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf61e21-0a83-4b38-ab4c-8959c74bdc1d service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:08.892781 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf61e21-0a83-4b38-ab4c-8959c74bdc1d service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:08.893060 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf61e21-0a83-4b38-ab4c-8959c74bdc1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:08.904480 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf61e21-0a83-4b38-ab4c-8959c74bdc1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:08.906021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf61e21-0a83-4b38-ab4c-8959c74bdc1d service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:08.918247 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1083/2172] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:54:09.223621 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8280ac7-d1e6-49b7-b446-1851cb6793d5 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:09.225191 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d8280ac7-d1e6-49b7-b446-1851cb6793d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:09.241587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8280ac7-d1e6-49b7-b446-1851cb6793d5 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:09.246980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:09.247906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:09.248281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:09.248502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] PATH_INFO: `/v3/projects/0fb227a492654ab4a7f9ff1df08c510f` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:09.249810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb227a492654ab4a7f9ff1df08c510f)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:09.253723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:09.306376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb227a492654ab4a7f9ff1df08c510f', 'request_id': 'req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:09.317652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb227a492654ab4a7f9ff1df08c510f', 'request_id': 'req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:09.324653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb227a492654ab4a7f9ff1df08c510f', 'request_id': 'req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:09.326683 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:09.338851 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] The token cache is being invalidate because project 0fb227a492654ab4a7f9ff1df08c510f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:54:09.339138 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:09.339810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:09.341397 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1090/2173] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:54:09 2022] DELETE /identity/v3/projects/0fb227a492654ab4a7f9ff1df08c510f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:09.345474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf61e21-0a83-4b38-ab4c-8959c74bdc1d service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:09.356984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf61e21-0a83-4b38-ab4c-8959c74bdc1d service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:09.390784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:09.418229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bdaa92-55f8-4bad-84f8-c57dcdc15bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:09.419125 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bdaa92-55f8-4bad-84f8-c57dcdc15bb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:09.419413 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bdaa92-55f8-4bad-84f8-c57dcdc15bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:09.419693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bdaa92-55f8-4bad-84f8-c57dcdc15bb4 admin admin] PATH_INFO: `/v3/users/d1dfce23a5e64316bd65d3183ccf5eb2` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:09.421019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bdaa92-55f8-4bad-84f8-c57dcdc15bb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1dfce23a5e64316bd65d3183ccf5eb2)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:09.422979 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bdaa92-55f8-4bad-84f8-c57dcdc15bb4 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:09.481614 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d2bdaa92-55f8-4bad-84f8-c57dcdc15bb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1dfce23a5e64316bd65d3183ccf5eb2'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:09.498833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d2bdaa92-55f8-4bad-84f8-c57dcdc15bb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1dfce23a5e64316bd65d3183ccf5eb2'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:09.499586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d2bdaa92-55f8-4bad-84f8-c57dcdc15bb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1dfce23a5e64316bd65d3183ccf5eb2'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:09.510302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d2bdaa92-55f8-4bad-84f8-c57dcdc15bb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1dfce23a5e64316bd65d3183ccf5eb2'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:09.511117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d2bdaa92-55f8-4bad-84f8-c57dcdc15bb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1dfce23a5e64316bd65d3183ccf5eb2'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:09.524880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1084/2174] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:54:09 2022] DELETE /identity/v3/users/d1dfce23a5e64316bd65d3183ccf5eb2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:10.143114 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:10.144688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:10.159659 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a93f1-f0d8-4cc3-a10c-6bc00d40bedc admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:10.242946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb21557-306a-4d37-a3df-773f823f5cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:10.243904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb21557-306a-4d37-a3df-773f823f5cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:10.244233 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb21557-306a-4d37-a3df-773f823f5cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:10.244567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb21557-306a-4d37-a3df-773f823f5cf8 admin admin] PATH_INFO: `/v3/projects/a96aa24a3ce640c9b926e3f9a7af4fb7` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:10.245830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb21557-306a-4d37-a3df-773f823f5cf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a96aa24a3ce640c9b926e3f9a7af4fb7)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:10.247784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb21557-306a-4d37-a3df-773f823f5cf8 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:10.311879 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3fb21557-306a-4d37-a3df-773f823f5cf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a96aa24a3ce640c9b926e3f9a7af4fb7', 'request_id': 'req-3fb21557-306a-4d37-a3df-773f823f5cf8'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:10.322260 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3fb21557-306a-4d37-a3df-773f823f5cf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a96aa24a3ce640c9b926e3f9a7af4fb7', 'request_id': 'req-3fb21557-306a-4d37-a3df-773f823f5cf8'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:10.332719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3fb21557-306a-4d37-a3df-773f823f5cf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a96aa24a3ce640c9b926e3f9a7af4fb7', 'request_id': 'req-3fb21557-306a-4d37-a3df-773f823f5cf8'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:10.359909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3fb21557-306a-4d37-a3df-773f823f5cf8 admin admin] The token cache is being invalidate because project a96aa24a3ce640c9b926e3f9a7af4fb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:54:10.360232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3fb21557-306a-4d37-a3df-773f823f5cf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:10.361075 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3fb21557-306a-4d37-a3df-773f823f5cf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:10.362721 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1091/2175] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:54:10 2022] DELETE /identity/v3/projects/a96aa24a3ce640c9b926e3f9a7af4fb7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:10.367637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d501e-0a11-4474-82ba-44b880248f97 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:10.368602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d501e-0a11-4474-82ba-44b880248f97 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:10.369058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d501e-0a11-4474-82ba-44b880248f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:10.413733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-754d501e-0a11-4474-82ba-44b880248f97 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:10.415264 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-754d501e-0a11-4474-82ba-44b880248f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:10.485419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1085/2176] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:10.493947 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655781c0-5939-448e-bbfa-7333b31c29e1 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:10.494239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655781c0-5939-448e-bbfa-7333b31c29e1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:10.494522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655781c0-5939-448e-bbfa-7333b31c29e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:10.513574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-655781c0-5939-448e-bbfa-7333b31c29e1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:10.514783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-655781c0-5939-448e-bbfa-7333b31c29e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:10.523034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1092/2177] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:10.527407 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d501e-0a11-4474-82ba-44b880248f97 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:10.537371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d501e-0a11-4474-82ba-44b880248f97 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:10.542032 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3e67d-eb83-462e-8e0e-a2c78b38a5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:10.542698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3e67d-eb83-462e-8e0e-a2c78b38a5b1 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:10.542986 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3e67d-eb83-462e-8e0e-a2c78b38a5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:10.543262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f3e67d-eb83-462e-8e0e-a2c78b38a5b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:10.544179 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f3e67d-eb83-462e-8e0e-a2c78b38a5b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:54:10.544444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f3e67d-eb83-462e-8e0e-a2c78b38a5b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:10.546240 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f3e67d-eb83-462e-8e0e-a2c78b38a5b1 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:10.559957 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1086/2178] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:54:10.563292 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655781c0-5939-448e-bbfa-7333b31c29e1 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:10.568686 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3e67d-eb83-462e-8e0e-a2c78b38a5b1 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:10.570421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c9f3e67d-eb83-462e-8e0e-a2c78b38a5b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:10.574189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655781c0-5939-448e-bbfa-7333b31c29e1 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:10.578783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dbda5-467c-40f9-8bbe-5dcf3d9a7c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:10.579470 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dbda5-467c-40f9-8bbe-5dcf3d9a7c34 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:10.579756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dbda5-467c-40f9-8bbe-5dcf3d9a7c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:10.580071 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85dbda5-467c-40f9-8bbe-5dcf3d9a7c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:10.580769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85dbda5-467c-40f9-8bbe-5dcf3d9a7c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:10.582676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85dbda5-467c-40f9-8bbe-5dcf3d9a7c34 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:10.583482 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f3e67d-eb83-462e-8e0e-a2c78b38a5b1 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:10.589603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639fcea-ddce-4413-a0fc-950a8df8ce64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:10.590478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639fcea-ddce-4413-a0fc-950a8df8ce64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:10.590763 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639fcea-ddce-4413-a0fc-950a8df8ce64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:10.591040 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639fcea-ddce-4413-a0fc-950a8df8ce64 admin admin] PATH_INFO: `/v3/users/c3e557927b1249e49f793815650f1345` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:10.592281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639fcea-ddce-4413-a0fc-950a8df8ce64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3e557927b1249e49f793815650f1345)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:10.594181 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639fcea-ddce-4413-a0fc-950a8df8ce64 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:10.615983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1093/2179] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:54:10 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:54:10.618929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dbda5-467c-40f9-8bbe-5dcf3d9a7c34 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:10.649411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:10.660863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a639fcea-ddce-4413-a0fc-950a8df8ce64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3e557927b1249e49f793815650f1345'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:10.673992 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a639fcea-ddce-4413-a0fc-950a8df8ce64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3e557927b1249e49f793815650f1345'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:10.674740 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a639fcea-ddce-4413-a0fc-950a8df8ce64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3e557927b1249e49f793815650f1345'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:10.675415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=90,keystone:DELETE=8,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:10.683543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a639fcea-ddce-4413-a0fc-950a8df8ce64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3e557927b1249e49f793815650f1345'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:10.684228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a639fcea-ddce-4413-a0fc-950a8df8ce64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3e557927b1249e49f793815650f1345'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:10.690646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85dbda5-467c-40f9-8bbe-5dcf3d9a7c34 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:10.695714 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a17a60-1fb4-49f2-9871-88caa5974965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:10.696912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a17a60-1fb4-49f2-9871-88caa5974965 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:10.697198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a17a60-1fb4-49f2-9871-88caa5974965 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:10.697473 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a17a60-1fb4-49f2-9871-88caa5974965 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:10.698158 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a17a60-1fb4-49f2-9871-88caa5974965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:10.698381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1087/2180] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:54:10 2022] DELETE /identity/v3/users/c3e557927b1249e49f793815650f1345 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:54:10.700243 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a17a60-1fb4-49f2-9871-88caa5974965 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:10.702903 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2aefe-dc98-4a20-aa58-0ba2324046a6 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:10.703196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2aefe-dc98-4a20-aa58-0ba2324046a6 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:10.703480 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb2aefe-dc98-4a20-aa58-0ba2324046a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:10.721879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-deb2aefe-dc98-4a20-aa58-0ba2324046a6 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:10.722920 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1094/2181] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:54:10 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:10.723056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-deb2aefe-dc98-4a20-aa58-0ba2324046a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:10.726601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a17a60-1fb4-49f2-9871-88caa5974965 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:10.728234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d8a17a60-1fb4-49f2-9871-88caa5974965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:10.741583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a17a60-1fb4-49f2-9871-88caa5974965 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:10.792978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1088/2182] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:54:10.805581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2aefe-dc98-4a20-aa58-0ba2324046a6 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:10.807183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-deb2aefe-dc98-4a20-aa58-0ba2324046a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:10.815367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7020bb9-aae0-463e-af64-cf372d1b5e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:10.816327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7020bb9-aae0-463e-af64-cf372d1b5e0c admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:10.816622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7020bb9-aae0-463e-af64-cf372d1b5e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:10.816884 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7020bb9-aae0-463e-af64-cf372d1b5e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:10.817827 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7020bb9-aae0-463e-af64-cf372d1b5e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:10.819796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7020bb9-aae0-463e-af64-cf372d1b5e0c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:10.832891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1095/2183] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:54:10.837427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7020bb9-aae0-463e-af64-cf372d1b5e0c admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:10.849259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7020bb9-aae0-463e-af64-cf372d1b5e0c admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:10.854457 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d1598-59a8-4835-9b60-22b9b71feed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:10.855229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d1598-59a8-4835-9b60-22b9b71feed8 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:10.855594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d1598-59a8-4835-9b60-22b9b71feed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:10.856099 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d1598-59a8-4835-9b60-22b9b71feed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:10.856914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d1598-59a8-4835-9b60-22b9b71feed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:10.858612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d1598-59a8-4835-9b60-22b9b71feed8 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:10.870320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1096/2184] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:10.874299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d1598-59a8-4835-9b60-22b9b71feed8 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:10.887045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb2aefe-dc98-4a20-aa58-0ba2324046a6 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:10.889082 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d1598-59a8-4835-9b60-22b9b71feed8 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:10.891427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1518e3e-2060-4878-85e1-792948940187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:10.892179 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1518e3e-2060-4878-85e1-792948940187 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:10.892469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1518e3e-2060-4878-85e1-792948940187 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:10.892822 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1518e3e-2060-4878-85e1-792948940187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:10.894088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8ca60-d137-4767-a4c1-e9465acf0410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:10.895050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8ca60-d137-4767-a4c1-e9465acf0410 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:10.895336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8ca60-d137-4767-a4c1-e9465acf0410 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:10.895653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8ca60-d137-4767-a4c1-e9465acf0410 admin admin] PATH_INFO: `/v3/projects/2afa2acbbc3c4e21ba7e211095bb592c/users/36637b05bded4ccfbfbaaaacc6ae38d7/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:10.904985 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1518e3e-2060-4878-85e1-792948940187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:10.906722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1518e3e-2060-4878-85e1-792948940187 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:10.914145 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8ca60-d137-4767-a4c1-e9465acf0410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2afa2acbbc3c4e21ba7e211095bb592c, user_id=36637b05bded4ccfbfbaaaacc6ae38d7, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:10.916818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8ca60-d137-4767-a4c1-e9465acf0410 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:10.918811 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1089/2185] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:54:10.933921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1097/2186] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:54:10 2022] PUT /identity/v3/projects/2afa2acbbc3c4e21ba7e211095bb592c/users/36637b05bded4ccfbfbaaaacc6ae38d7/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:54:10.945647 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842860c-d9f8-47f8-ba17-97e9423938ee None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:10.945647 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842860c-d9f8-47f8-ba17-97e9423938ee None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:10.945984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842860c-d9f8-47f8-ba17-97e9423938ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:10.982256 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-0842860c-d9f8-47f8-ba17-97e9423938ee None None] MFA Rules not processed for user `36637b05bded4ccfbfbaaaacc6ae38d7`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:10.983585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0842860c-d9f8-47f8-ba17-97e9423938ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:11.042220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:11.065655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1090/2187] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:11.079879 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8ca60-d137-4767-a4c1-e9465acf0410 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:11.081820 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-88b8ca60-d137-4767-a4c1-e9465acf0410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:11.184314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8ca60-d137-4767-a4c1-e9465acf0410 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:11.189854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f1f7a-64d7-4db4-8c86-05e7f4e21b85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:11.190076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f1f7a-64d7-4db4-8c86-05e7f4e21b85 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:11.190334 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f1f7a-64d7-4db4-8c86-05e7f4e21b85 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:11.190618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f1f7a-64d7-4db4-8c86-05e7f4e21b85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:11.200518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f1f7a-64d7-4db4-8c86-05e7f4e21b85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:11.202280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f1f7a-64d7-4db4-8c86-05e7f4e21b85 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:11.212866 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1098/2188] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:54:11.450610 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842860c-d9f8-47f8-ba17-97e9423938ee None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:11.453143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0842860c-d9f8-47f8-ba17-97e9423938ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:11.589703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842860c-d9f8-47f8-ba17-97e9423938ee None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:11.596591 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3774e3-1e97-41ea-9a32-c8eca530577a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:11.597592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3774e3-1e97-41ea-9a32-c8eca530577a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:11.597951 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3774e3-1e97-41ea-9a32-c8eca530577a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:11.598428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3774e3-1e97-41ea-9a32-c8eca530577a admin admin] PATH_INFO: `/v3/projects/6c04c69de4f746b98596b3538a77f57b` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:11.600293 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3774e3-1e97-41ea-9a32-c8eca530577a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c04c69de4f746b98596b3538a77f57b)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:11.602980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3774e3-1e97-41ea-9a32-c8eca530577a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:11.692458 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7b3774e3-1e97-41ea-9a32-c8eca530577a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c04c69de4f746b98596b3538a77f57b', 'request_id': 'req-7b3774e3-1e97-41ea-9a32-c8eca530577a'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:11.701572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7b3774e3-1e97-41ea-9a32-c8eca530577a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c04c69de4f746b98596b3538a77f57b', 'request_id': 'req-7b3774e3-1e97-41ea-9a32-c8eca530577a'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:11.720743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7b3774e3-1e97-41ea-9a32-c8eca530577a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c04c69de4f746b98596b3538a77f57b', 'request_id': 'req-7b3774e3-1e97-41ea-9a32-c8eca530577a'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:11.738772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7b3774e3-1e97-41ea-9a32-c8eca530577a admin admin] The token cache is being invalidate because project 6c04c69de4f746b98596b3538a77f57b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:54:11.739302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7b3774e3-1e97-41ea-9a32-c8eca530577a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:11.740082 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7b3774e3-1e97-41ea-9a32-c8eca530577a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:11.741904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1091/2189] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:54:11 2022] DELETE /identity/v3/projects/6c04c69de4f746b98596b3538a77f57b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:11.747115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bf193-0594-46c6-95cb-e122fb518f69 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:11.747457 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bf193-0594-46c6-95cb-e122fb518f69 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:11.747695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bf193-0594-46c6-95cb-e122fb518f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:11.767628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-420bf193-0594-46c6-95cb-e122fb518f69 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:11.769050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-420bf193-0594-46c6-95cb-e122fb518f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:11.819284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:11.840569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1099/2190] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:11.850275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74337c89-fbb4-465d-a406-c1c57c40c226 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:11.850573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74337c89-fbb4-465d-a406-c1c57c40c226 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:11.850857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74337c89-fbb4-465d-a406-c1c57c40c226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:11.871504 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-74337c89-fbb4-465d-a406-c1c57c40c226 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:11.873062 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-74337c89-fbb4-465d-a406-c1c57c40c226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:11.881587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1092/2191] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:11.886119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bf193-0594-46c6-95cb-e122fb518f69 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:11.896793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bf193-0594-46c6-95cb-e122fb518f69 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:11.901427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4b361-a646-464b-90dc-84b29601f8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:11.902095 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4b361-a646-464b-90dc-84b29601f8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:11.902381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4b361-a646-464b-90dc-84b29601f8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:11.902785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4b361-a646-464b-90dc-84b29601f8d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:11.903538 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4b361-a646-464b-90dc-84b29601f8d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:54:11.903852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4b361-a646-464b-90dc-84b29601f8d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:11.905505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b4b361-a646-464b-90dc-84b29601f8d4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:11.919960 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1100/2192] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:54:11.923366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74337c89-fbb4-465d-a406-c1c57c40c226 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:11.932566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74337c89-fbb4-465d-a406-c1c57c40c226 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:11.940245 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60025c5-f781-4c03-a7d8-4aabd88674fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:11.940914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60025c5-f781-4c03-a7d8-4aabd88674fe admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:11.941199 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60025c5-f781-4c03-a7d8-4aabd88674fe admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:11.941476 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60025c5-f781-4c03-a7d8-4aabd88674fe admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:11.942174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60025c5-f781-4c03-a7d8-4aabd88674fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:11.944113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60025c5-f781-4c03-a7d8-4aabd88674fe admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:11.973552 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1093/2193] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:54:11 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:11.975143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4b361-a646-464b-90dc-84b29601f8d4 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:11.985237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b4b361-a646-464b-90dc-84b29601f8d4 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:12.043204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ba13e-9c9d-4a04-8bcc-f19d878d832a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:12.043880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ba13e-9c9d-4a04-8bcc-f19d878d832a admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:12.044234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ba13e-9c9d-4a04-8bcc-f19d878d832a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:12.044527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ba13e-9c9d-4a04-8bcc-f19d878d832a admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:12.045311 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ba13e-9c9d-4a04-8bcc-f19d878d832a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:12.047531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ba13e-9c9d-4a04-8bcc-f19d878d832a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:12.068544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1101/2194] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:54:11 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:12.073095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60025c5-f781-4c03-a7d8-4aabd88674fe admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:12.083746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60025c5-f781-4c03-a7d8-4aabd88674fe admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:12.088437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711af0d-f59c-4227-b659-c046aa9aa2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:12.089208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711af0d-f59c-4227-b659-c046aa9aa2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:12.089374 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711af0d-f59c-4227-b659-c046aa9aa2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:12.089652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1711af0d-f59c-4227-b659-c046aa9aa2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:12.090393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711af0d-f59c-4227-b659-c046aa9aa2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:12.092065 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1711af0d-f59c-4227-b659-c046aa9aa2c4 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:12.103835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1094/2195] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:12.107837 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ba13e-9c9d-4a04-8bcc-f19d878d832a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:12.118212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ba13e-9c9d-4a04-8bcc-f19d878d832a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:12.123032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693d4d2-f827-42b6-a497-0ad7da626979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:12.123707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693d4d2-f827-42b6-a497-0ad7da626979 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:12.124026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693d4d2-f827-42b6-a497-0ad7da626979 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:12.124308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693d4d2-f827-42b6-a497-0ad7da626979 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:12.125040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693d4d2-f827-42b6-a497-0ad7da626979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:12.126717 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693d4d2-f827-42b6-a497-0ad7da626979 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:12.139166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1102/2196] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:12.144652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711af0d-f59c-4227-b659-c046aa9aa2c4 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:12.155223 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1711af0d-f59c-4227-b659-c046aa9aa2c4 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:12.160664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bf0cd-092a-4b85-bdb3-ab0c9c1b2709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:12.163964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bf0cd-092a-4b85-bdb3-ab0c9c1b2709 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:12.163964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bf0cd-092a-4b85-bdb3-ab0c9c1b2709 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:12.163964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bf0cd-092a-4b85-bdb3-ab0c9c1b2709 admin admin] PATH_INFO: `/v3/projects/a84353cfe41640e3b13d561b615e1142/users/d3b3a526943646dcba7bb1806601696b/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:12.186314 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bf0cd-092a-4b85-bdb3-ab0c9c1b2709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84353cfe41640e3b13d561b615e1142, user_id=d3b3a526943646dcba7bb1806601696b, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:12.189137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691bf0cd-092a-4b85-bdb3-ab0c9c1b2709 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:12.203539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1095/2197] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:54:12 2022] PUT /identity/v3/projects/a84353cfe41640e3b13d561b615e1142/users/d3b3a526943646dcba7bb1806601696b/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:12.211446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793051e6-732e-4213-8dc1-ef1c83e5176d None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:12.211977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793051e6-732e-4213-8dc1-ef1c83e5176d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:12.212076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793051e6-732e-4213-8dc1-ef1c83e5176d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:12.236833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-793051e6-732e-4213-8dc1-ef1c83e5176d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:12.238184 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-793051e6-732e-4213-8dc1-ef1c83e5176d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:12.324610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1103/2198] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:12.336770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bf0cd-092a-4b85-bdb3-ab0c9c1b2709 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:12.338542 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-691bf0cd-092a-4b85-bdb3-ab0c9c1b2709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:12.380497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:12.461124 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bf0cd-092a-4b85-bdb3-ab0c9c1b2709 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:12.464913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476ceef-339e-49e9-b414-8ef9c980b875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:12.465573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476ceef-339e-49e9-b414-8ef9c980b875 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:12.465856 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476ceef-339e-49e9-b414-8ef9c980b875 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:12.466133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3476ceef-339e-49e9-b414-8ef9c980b875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:12.475849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476ceef-339e-49e9-b414-8ef9c980b875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:12.477411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3476ceef-339e-49e9-b414-8ef9c980b875 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:12.487753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1096/2199] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:54:13.218570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccfe32-a66c-4b87-adf5-4972b0d302df None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:13.219221 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccfe32-a66c-4b87-adf5-4972b0d302df None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:13.219221 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccfe32-a66c-4b87-adf5-4972b0d302df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:13.248548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-82ccfe32-a66c-4b87-adf5-4972b0d302df None None] MFA Rules not processed for user `d3b3a526943646dcba7bb1806601696b`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:13.249816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-82ccfe32-a66c-4b87-adf5-4972b0d302df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:13.298146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:13.325328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1104/2200] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:13.336580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476ceef-339e-49e9-b414-8ef9c980b875 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:13.338573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3476ceef-339e-49e9-b414-8ef9c980b875 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:13.400994 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3476ceef-339e-49e9-b414-8ef9c980b875 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:13.404685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2187a8-d1d9-4a39-a45c-51e640978a22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:13.405338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2187a8-d1d9-4a39-a45c-51e640978a22 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:13.405649 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2187a8-d1d9-4a39-a45c-51e640978a22 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:13.405926 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2187a8-d1d9-4a39-a45c-51e640978a22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:13.417371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2187a8-d1d9-4a39-a45c-51e640978a22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:13.418860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2187a8-d1d9-4a39-a45c-51e640978a22 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:13.429881 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1097/2201] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:54:14.858052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a50bc5-23f0-476a-99bb-ee0e2a1a17a0 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:14.858571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a50bc5-23f0-476a-99bb-ee0e2a1a17a0 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:14.858765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a50bc5-23f0-476a-99bb-ee0e2a1a17a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:14.879114 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-52a50bc5-23f0-476a-99bb-ee0e2a1a17a0 None None] MFA Rules not processed for user `d3b3a526943646dcba7bb1806601696b`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:14.880402 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-52a50bc5-23f0-476a-99bb-ee0e2a1a17a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:14.887606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1105/2202] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:15.620575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2187a8-d1d9-4a39-a45c-51e640978a22 service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:15.620575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1a2187a8-d1d9-4a39-a45c-51e640978a22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:15.635366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2187a8-d1d9-4a39-a45c-51e640978a22 service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:15.639846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988d11e-c8f4-4e2b-b22d-e9389b63da2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:15.640768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988d11e-c8f4-4e2b-b22d-e9389b63da2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:15.641049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988d11e-c8f4-4e2b-b22d-e9389b63da2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:15.641325 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988d11e-c8f4-4e2b-b22d-e9389b63da2b admin admin] PATH_INFO: `/v3/users/36637b05bded4ccfbfbaaaacc6ae38d7` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:15.643056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988d11e-c8f4-4e2b-b22d-e9389b63da2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=36637b05bded4ccfbfbaaaacc6ae38d7)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:15.645019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988d11e-c8f4-4e2b-b22d-e9389b63da2b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:15.706397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1988d11e-c8f4-4e2b-b22d-e9389b63da2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36637b05bded4ccfbfbaaaacc6ae38d7'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:15.720929 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1988d11e-c8f4-4e2b-b22d-e9389b63da2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36637b05bded4ccfbfbaaaacc6ae38d7'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:15.721760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1988d11e-c8f4-4e2b-b22d-e9389b63da2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36637b05bded4ccfbfbaaaacc6ae38d7'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:15.731035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1988d11e-c8f4-4e2b-b22d-e9389b63da2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36637b05bded4ccfbfbaaaacc6ae38d7'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:15.731741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-1988d11e-c8f4-4e2b-b22d-e9389b63da2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36637b05bded4ccfbfbaaaacc6ae38d7'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:15.748271 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1098/2203] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:54:15 2022] DELETE /identity/v3/users/36637b05bded4ccfbfbaaaacc6ae38d7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:15.753382 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59406af9-5d62-400c-a635-6ab64e02e4bd None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:15.753676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59406af9-5d62-400c-a635-6ab64e02e4bd None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:15.753955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59406af9-5d62-400c-a635-6ab64e02e4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:15.782459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-59406af9-5d62-400c-a635-6ab64e02e4bd None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:15.784245 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-59406af9-5d62-400c-a635-6ab64e02e4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:15.862121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1106/2204] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:54:15.875443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988d11e-c8f4-4e2b-b22d-e9389b63da2b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:15.877202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1988d11e-c8f4-4e2b-b22d-e9389b63da2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:15.978887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:16.022495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988d11e-c8f4-4e2b-b22d-e9389b63da2b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:16.031963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10da4e6-ccd3-419b-a142-509305fd7441 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:16.039664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10da4e6-ccd3-419b-a142-509305fd7441 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:16.039664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10da4e6-ccd3-419b-a142-509305fd7441 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:16.039664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10da4e6-ccd3-419b-a142-509305fd7441 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:16.045698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10da4e6-ccd3-419b-a142-509305fd7441 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:16.047685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10da4e6-ccd3-419b-a142-509305fd7441 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:16.077828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1099/2205] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:54:16.093525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59406af9-5d62-400c-a635-6ab64e02e4bd None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:16.096629 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-59406af9-5d62-400c-a635-6ab64e02e4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:16.116669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59406af9-5d62-400c-a635-6ab64e02e4bd None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:16.126585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2795a3-5f62-48cd-9c20-67d293f5ecaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:16.128175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2795a3-5f62-48cd-9c20-67d293f5ecaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:16.128622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2795a3-5f62-48cd-9c20-67d293f5ecaf admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:16.129198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2795a3-5f62-48cd-9c20-67d293f5ecaf admin admin] PATH_INFO: `/v3/users/f7cc4009257d4a38bcce57f12f9d5265` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:16.130705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2795a3-5f62-48cd-9c20-67d293f5ecaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7cc4009257d4a38bcce57f12f9d5265)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:16.134194 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2795a3-5f62-48cd-9c20-67d293f5ecaf admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:16.214776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-de2795a3-5f62-48cd-9c20-67d293f5ecaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7cc4009257d4a38bcce57f12f9d5265'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:16.229751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-de2795a3-5f62-48cd-9c20-67d293f5ecaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7cc4009257d4a38bcce57f12f9d5265'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:16.239271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-de2795a3-5f62-48cd-9c20-67d293f5ecaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7cc4009257d4a38bcce57f12f9d5265'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:16.239955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-de2795a3-5f62-48cd-9c20-67d293f5ecaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7cc4009257d4a38bcce57f12f9d5265'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:16.250053 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-de2795a3-5f62-48cd-9c20-67d293f5ecaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7cc4009257d4a38bcce57f12f9d5265'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:16.262991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1107/2206] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:54:16 2022] DELETE /identity/v3/users/f7cc4009257d4a38bcce57f12f9d5265 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:16.527420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10da4e6-ccd3-419b-a142-509305fd7441 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:16.574853 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10da4e6-ccd3-419b-a142-509305fd7441 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:16.715828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:16.716921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:16.717460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:16.717877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] PATH_INFO: `/v3/projects/230223aca78741fd81f406bdfcd09fe7` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:16.719885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=230223aca78741fd81f406bdfcd09fe7)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:16.723386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:16.807088 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '230223aca78741fd81f406bdfcd09fe7', 'request_id': 'req-06612536-2e08-41cd-abff-959ab7df2bfc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:16.824907 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '230223aca78741fd81f406bdfcd09fe7', 'request_id': 'req-06612536-2e08-41cd-abff-959ab7df2bfc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:16.840484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '230223aca78741fd81f406bdfcd09fe7', 'request_id': 'req-06612536-2e08-41cd-abff-959ab7df2bfc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:16.873470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] The token cache is being invalidate because project 230223aca78741fd81f406bdfcd09fe7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:54:16.873923 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:16.875594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:16.886109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1100/2207] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:54:16 2022] DELETE /identity/v3/projects/230223aca78741fd81f406bdfcd09fe7 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:16.888040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2795a3-5f62-48cd-9c20-67d293f5ecaf admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:16.890835 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-de2795a3-5f62-48cd-9c20-67d293f5ecaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:16.907304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2795a3-5f62-48cd-9c20-67d293f5ecaf admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:16.976531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:16.978917 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:17.018088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:17.073633 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2361743b-bbca-4e03-8f38-3562c493a999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:17.075142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361743b-bbca-4e03-8f38-3562c493a999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:17.075718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361743b-bbca-4e03-8f38-3562c493a999 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:17.076212 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2361743b-bbca-4e03-8f38-3562c493a999 admin admin] PATH_INFO: `/v3/users/3f51ffe8c8df4e65ac3068d9c6ce9a0a` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:17.080433 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361743b-bbca-4e03-8f38-3562c493a999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f51ffe8c8df4e65ac3068d9c6ce9a0a)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:17.086535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2361743b-bbca-4e03-8f38-3562c493a999 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:17.199747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06612536-2e08-41cd-abff-959ab7df2bfc admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:17.209657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191b393-3049-4ae7-80e0-7186307136c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:17.211236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191b393-3049-4ae7-80e0-7186307136c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:17.211673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191b393-3049-4ae7-80e0-7186307136c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:17.214447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191b393-3049-4ae7-80e0-7186307136c9 admin admin] PATH_INFO: `/v3/projects/2afa2acbbc3c4e21ba7e211095bb592c` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:17.216985 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191b393-3049-4ae7-80e0-7186307136c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2afa2acbbc3c4e21ba7e211095bb592c)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:17.219450 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3191b393-3049-4ae7-80e0-7186307136c9 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:17.227492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-2361743b-bbca-4e03-8f38-3562c493a999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f51ffe8c8df4e65ac3068d9c6ce9a0a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.230698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:17.253443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-2361743b-bbca-4e03-8f38-3562c493a999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f51ffe8c8df4e65ac3068d9c6ce9a0a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.270325 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-2361743b-bbca-4e03-8f38-3562c493a999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f51ffe8c8df4e65ac3068d9c6ce9a0a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.273160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-2361743b-bbca-4e03-8f38-3562c493a999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f51ffe8c8df4e65ac3068d9c6ce9a0a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.286875 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-2361743b-bbca-4e03-8f38-3562c493a999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f51ffe8c8df4e65ac3068d9c6ce9a0a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.295790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1108/2208] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:54:16 2022] DELETE /identity/v3/users/3f51ffe8c8df4e65ac3068d9c6ce9a0a => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:17.305378 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3191b393-3049-4ae7-80e0-7186307136c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2afa2acbbc3c4e21ba7e211095bb592c', 'request_id': 'req-3191b393-3049-4ae7-80e0-7186307136c9'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.322257 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3191b393-3049-4ae7-80e0-7186307136c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2afa2acbbc3c4e21ba7e211095bb592c', 'request_id': 'req-3191b393-3049-4ae7-80e0-7186307136c9'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.332551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3191b393-3049-4ae7-80e0-7186307136c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2afa2acbbc3c4e21ba7e211095bb592c', 'request_id': 'req-3191b393-3049-4ae7-80e0-7186307136c9'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.374510 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3191b393-3049-4ae7-80e0-7186307136c9 admin admin] The token cache is being invalidate because project 2afa2acbbc3c4e21ba7e211095bb592c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:54:17.375073 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3191b393-3049-4ae7-80e0-7186307136c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.376319 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3191b393-3049-4ae7-80e0-7186307136c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.378768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1101/2209] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:54:16 2022] DELETE /identity/v3/projects/2afa2acbbc3c4e21ba7e211095bb592c => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:54:17.383753 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6989844-8c06-414d-b243-1593501ee684 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:17.384165 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6989844-8c06-414d-b243-1593501ee684 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:17.384428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6989844-8c06-414d-b243-1593501ee684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:17.408386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f6989844-8c06-414d-b243-1593501ee684 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:17.409899 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f6989844-8c06-414d-b243-1593501ee684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:17.485689 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1109/2210] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:54:17.494463 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910197f8-0e4d-452f-a376-9410bd4409db None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:17.494740 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910197f8-0e4d-452f-a376-9410bd4409db None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:17.495017 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910197f8-0e4d-452f-a376-9410bd4409db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:17.521701 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-910197f8-0e4d-452f-a376-9410bd4409db None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:17.523070 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-910197f8-0e4d-452f-a376-9410bd4409db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:17.532674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1102/2211] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:17.537208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6989844-8c06-414d-b243-1593501ee684 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:17.544046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910197f8-0e4d-452f-a376-9410bd4409db None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:17.545667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-910197f8-0e4d-452f-a376-9410bd4409db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:17.554380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6989844-8c06-414d-b243-1593501ee684 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:17.560617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55c915-d75c-464a-a6b4-f2087a747cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:17.561311 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55c915-d75c-464a-a6b4-f2087a747cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:17.561596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55c915-d75c-464a-a6b4-f2087a747cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:17.561876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55c915-d75c-464a-a6b4-f2087a747cc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:17.562622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55c915-d75c-464a-a6b4-f2087a747cc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:54:17.562985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55c915-d75c-464a-a6b4-f2087a747cc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:17.565230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55c915-d75c-464a-a6b4-f2087a747cc9 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:17.572884 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910197f8-0e4d-452f-a376-9410bd4409db None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:17.579363 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff85ab-444e-4328-915e-08b15a8914a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:17.581060 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff85ab-444e-4328-915e-08b15a8914a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:17.581501 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff85ab-444e-4328-915e-08b15a8914a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:17.581933 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff85ab-444e-4328-915e-08b15a8914a3 admin admin] PATH_INFO: `/v3/projects/000494f9d30e46978b4595905221a272` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:17.583324 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff85ab-444e-4328-915e-08b15a8914a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=000494f9d30e46978b4595905221a272)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:17.585424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff85ab-444e-4328-915e-08b15a8914a3 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:17.588998 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1110/2212] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:54:17.596424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55c915-d75c-464a-a6b4-f2087a747cc9 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:17.614062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55c915-d75c-464a-a6b4-f2087a747cc9 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:17.631799 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1509d78-7414-4ee9-ac0e-9b439f7d5d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:17.632838 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1509d78-7414-4ee9-ac0e-9b439f7d5d52 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:17.640809 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1509d78-7414-4ee9-ac0e-9b439f7d5d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:17.642187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1509d78-7414-4ee9-ac0e-9b439f7d5d52 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:17.643472 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1509d78-7414-4ee9-ac0e-9b439f7d5d52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:17.649027 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1509d78-7414-4ee9-ac0e-9b439f7d5d52 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:17.687267 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-91ff85ab-444e-4328-915e-08b15a8914a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '000494f9d30e46978b4595905221a272', 'request_id': 'req-91ff85ab-444e-4328-915e-08b15a8914a3'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.701794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1111/2213] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:54:17 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:17.702384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-91ff85ab-444e-4328-915e-08b15a8914a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '000494f9d30e46978b4595905221a272', 'request_id': 'req-91ff85ab-444e-4328-915e-08b15a8914a3'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.707203 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1509d78-7414-4ee9-ac0e-9b439f7d5d52 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:17.712636 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-91ff85ab-444e-4328-915e-08b15a8914a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '000494f9d30e46978b4595905221a272', 'request_id': 'req-91ff85ab-444e-4328-915e-08b15a8914a3'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.728559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-91ff85ab-444e-4328-915e-08b15a8914a3 admin admin] The token cache is being invalidate because project 000494f9d30e46978b4595905221a272 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:54:17.729101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-91ff85ab-444e-4328-915e-08b15a8914a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.729301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-91ff85ab-444e-4328-915e-08b15a8914a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:17.732628 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1103/2214] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:54:17 2022] DELETE /identity/v3/projects/000494f9d30e46978b4595905221a272 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:54:17.740072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33185b-a72c-4b68-b3fd-7b527b432d87 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:17.759609 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33185b-a72c-4b68-b3fd-7b527b432d87 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:17.763779 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33185b-a72c-4b68-b3fd-7b527b432d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:17.812766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4a33185b-a72c-4b68-b3fd-7b527b432d87 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:17.816349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4a33185b-a72c-4b68-b3fd-7b527b432d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:17.823497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:17.952234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1509d78-7414-4ee9-ac0e-9b439f7d5d52 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:17.962910 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1104/2215] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:17.977364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a61c7-9d42-4290-ac4d-85c1c245d63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:17.978724 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a61c7-9d42-4290-ac4d-85c1c245d63e admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:17.979266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2f4fd-4f71-4b74-a1d9-a267136e62e5 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:17.979677 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a61c7-9d42-4290-ac4d-85c1c245d63e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:17.979947 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2f4fd-4f71-4b74-a1d9-a267136e62e5 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:17.980844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2f4fd-4f71-4b74-a1d9-a267136e62e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:17.990812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a61c7-9d42-4290-ac4d-85c1c245d63e admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:17.994571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a61c7-9d42-4290-ac4d-85c1c245d63e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:17.997974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0a61c7-9d42-4290-ac4d-85c1c245d63e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:18.010611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-b4f2f4fd-4f71-4b74-a1d9-a267136e62e5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:18.013261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b4f2f4fd-4f71-4b74-a1d9-a267136e62e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:18.026644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1112/2216] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:54:17 2022] POST /identity/v3/users => generated 293 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:18.031708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a61c7-9d42-4290-ac4d-85c1c245d63e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:18.035184 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8e0a61c7-9d42-4290-ac4d-85c1c245d63e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:18.039176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1105/2217] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:18.046007 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2f4fd-4f71-4b74-a1d9-a267136e62e5 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:18.054495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:18.060419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a61c7-9d42-4290-ac4d-85c1c245d63e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:18.066483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00328af-4d98-400c-8c71-f56e28a745e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:18.067593 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00328af-4d98-400c-8c71-f56e28a745e4 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:18.068103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00328af-4d98-400c-8c71-f56e28a745e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:18.068621 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00328af-4d98-400c-8c71-f56e28a745e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:18.069616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00328af-4d98-400c-8c71-f56e28a745e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:18.071508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00328af-4d98-400c-8c71-f56e28a745e4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:18.077341 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2f4fd-4f71-4b74-a1d9-a267136e62e5 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:18.085351 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bf211-0b28-4af0-ab32-f6d54c5ff30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:18.086583 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bf211-0b28-4af0-ab32-f6d54c5ff30e admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:18.087043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bf211-0b28-4af0-ab32-f6d54c5ff30e admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:18.087456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030bf211-0b28-4af0-ab32-f6d54c5ff30e admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:18.088557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bf211-0b28-4af0-ab32-f6d54c5ff30e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:54:18.089167 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bf211-0b28-4af0-ab32-f6d54c5ff30e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:18.089781 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1113/2218] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:54:18.090870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00328af-4d98-400c-8c71-f56e28a745e4 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:18.091781 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030bf211-0b28-4af0-ab32-f6d54c5ff30e admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:18.104196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00328af-4d98-400c-8c71-f56e28a745e4 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:18.116371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f521a6-bfa0-407c-bb18-0503974717a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:18.117381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f521a6-bfa0-407c-bb18-0503974717a9 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:18.117729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f521a6-bfa0-407c-bb18-0503974717a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:18.118075 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f521a6-bfa0-407c-bb18-0503974717a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:18.120749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f521a6-bfa0-407c-bb18-0503974717a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:18.122666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f521a6-bfa0-407c-bb18-0503974717a9 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:18.125586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1106/2219] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:54:18.130914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bf211-0b28-4af0-ab32-f6d54c5ff30e admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:18.144053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030bf211-0b28-4af0-ab32-f6d54c5ff30e admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:18.154444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8e0a1-7a5e-4a5d-aad6-d485f3a46f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:18.154647 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8e0a1-7a5e-4a5d-aad6-d485f3a46f6e admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:18.154913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8e0a1-7a5e-4a5d-aad6-d485f3a46f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:18.155190 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec8e0a1-7a5e-4a5d-aad6-d485f3a46f6e admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:18.155967 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1114/2220] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:18.156243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8e0a1-7a5e-4a5d-aad6-d485f3a46f6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:18.158352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec8e0a1-7a5e-4a5d-aad6-d485f3a46f6e admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:18.165335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f521a6-bfa0-407c-bb18-0503974717a9 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:18.195090 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f521a6-bfa0-407c-bb18-0503974717a9 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:18.203612 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1107/2221] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:54:18 2022] POST /identity/v3/projects => generated 375 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:18.203977 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8e0a1-7a5e-4a5d-aad6-d485f3a46f6e admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:18.351225 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7adc9e-64de-41de-a7cf-09e1d7c0f64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:18.352331 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7adc9e-64de-41de-a7cf-09e1d7c0f64d admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:18.352622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7adc9e-64de-41de-a7cf-09e1d7c0f64d admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:18.352903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7adc9e-64de-41de-a7cf-09e1d7c0f64d admin admin] PATH_INFO: `/v3/projects/af8b81470a69459fbd370a371672dfb4/users/6792a77be427458197c716454a3a8414/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:18.359589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec8e0a1-7a5e-4a5d-aad6-d485f3a46f6e admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:18.381460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83b300-5e8e-4981-9366-a1889df6e5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:18.382570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7adc9e-64de-41de-a7cf-09e1d7c0f64d admin admin] RBAC: Authorizing `identity:create_grant(project_id=af8b81470a69459fbd370a371672dfb4, user_id=6792a77be427458197c716454a3a8414, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:18.383468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83b300-5e8e-4981-9366-a1889df6e5c5 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:18.387290 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7adc9e-64de-41de-a7cf-09e1d7c0f64d admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:18.387619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83b300-5e8e-4981-9366-a1889df6e5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:18.388236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b83b300-5e8e-4981-9366-a1889df6e5c5 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:18.389330 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83b300-5e8e-4981-9366-a1889df6e5c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:18.409435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b83b300-5e8e-4981-9366-a1889df6e5c5 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:18.444023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1115/2222] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:54:18 2022] PUT /identity/v3/projects/af8b81470a69459fbd370a371672dfb4/users/6792a77be427458197c716454a3a8414/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:18.450785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883a1fa-20d5-47b4-aad7-d56e50d43f75 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:18.451084 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883a1fa-20d5-47b4-aad7-d56e50d43f75 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:18.451374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9883a1fa-20d5-47b4-aad7-d56e50d43f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:18.457114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1108/2223] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:54:18 2022] POST /identity/v3/users => generated 288 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:18.465271 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83b300-5e8e-4981-9366-a1889df6e5c5 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:18.480642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b83b300-5e8e-4981-9366-a1889df6e5c5 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:18.531061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-9883a1fa-20d5-47b4-aad7-d56e50d43f75 None None] MFA Rules not processed for user `6792a77be427458197c716454a3a8414`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:18.532414 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9883a1fa-20d5-47b4-aad7-d56e50d43f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:18.605586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8bf139-29f4-439c-869c-3b06b8c319ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:18.606564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bf139-29f4-439c-869c-3b06b8c319ab admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:18.606864 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bf139-29f4-439c-869c-3b06b8c319ab admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:18.607152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8bf139-29f4-439c-869c-3b06b8c319ab admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:18.608093 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8bf139-29f4-439c-869c-3b06b8c319ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:18.610046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8bf139-29f4-439c-869c-3b06b8c319ab admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:18.666514 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1109/2224] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:18.667539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8bf139-29f4-439c-869c-3b06b8c319ab admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:18.692181 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8bf139-29f4-439c-869c-3b06b8c319ab admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:18.693554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1116/2225] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:54:18.699115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883a1fa-20d5-47b4-aad7-d56e50d43f75 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:18.705289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbc045-44e0-4c29-8a5a-0100cacf55bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:18.706373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbc045-44e0-4c29-8a5a-0100cacf55bb admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:18.706666 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbc045-44e0-4c29-8a5a-0100cacf55bb admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:18.706948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbc045-44e0-4c29-8a5a-0100cacf55bb admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:18.707703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbc045-44e0-4c29-8a5a-0100cacf55bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:18.710633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbc045-44e0-4c29-8a5a-0100cacf55bb admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:18.715757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9883a1fa-20d5-47b4-aad7-d56e50d43f75 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:18.725111 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1110/2226] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:18.728878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e2a83-beb2-4e23-b25a-1f73de4d9618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:18.729686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e2a83-beb2-4e23-b25a-1f73de4d9618 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:18.729979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e2a83-beb2-4e23-b25a-1f73de4d9618 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:18.730258 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e2a83-beb2-4e23-b25a-1f73de4d9618 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:18.730684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbc045-44e0-4c29-8a5a-0100cacf55bb admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:18.731450 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e2a83-beb2-4e23-b25a-1f73de4d9618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:18.733153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2e2a83-beb2-4e23-b25a-1f73de4d9618 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:18.767106 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:18.774999 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbc045-44e0-4c29-8a5a-0100cacf55bb admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:18.795440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1117/2227] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:54:18 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:18.800666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e2a83-beb2-4e23-b25a-1f73de4d9618 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:18.803247 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e88a1-089e-4ed4-b55a-3825bcbb16bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:18.807250 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e88a1-089e-4ed4-b55a-3825bcbb16bf admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:18.807754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e88a1-089e-4ed4-b55a-3825bcbb16bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:18.808496 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e88a1-089e-4ed4-b55a-3825bcbb16bf admin admin] PATH_INFO: `/v3/projects/ba0533a36cbc4887b71ff0b351ce5996/users/bb3d8225603e411e847581fec95a16bb/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:18.829922 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e88a1-089e-4ed4-b55a-3825bcbb16bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba0533a36cbc4887b71ff0b351ce5996, user_id=bb3d8225603e411e847581fec95a16bb, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:18.833078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e88a1-089e-4ed4-b55a-3825bcbb16bf admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:18.892379 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1111/2228] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:54:18 2022] PUT /identity/v3/projects/ba0533a36cbc4887b71ff0b351ce5996/users/bb3d8225603e411e847581fec95a16bb/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:18.940740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e2a83-beb2-4e23-b25a-1f73de4d9618 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:18.958406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67709409-074f-4c80-b69c-37c8bcfb4a49 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:18.958406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67709409-074f-4c80-b69c-37c8bcfb4a49 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:18.958674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67709409-074f-4c80-b69c-37c8bcfb4a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:19.011452 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-67709409-074f-4c80-b69c-37c8bcfb4a49 None None] MFA Rules not processed for user `bb3d8225603e411e847581fec95a16bb`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:19.013109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-67709409-074f-4c80-b69c-37c8bcfb4a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:19.025380 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:19.059746 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822d177-19d7-490c-ab46-e0b426a78dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:19.060500 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822d177-19d7-490c-ab46-e0b426a78dad admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:19.060790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822d177-19d7-490c-ab46-e0b426a78dad admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:19.061071 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a822d177-19d7-490c-ab46-e0b426a78dad admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:19.061778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822d177-19d7-490c-ab46-e0b426a78dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:19.063742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a822d177-19d7-490c-ab46-e0b426a78dad admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:19.096837 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1118/2229] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:54:18 2022] POST /identity/v3/users => generated 293 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:19.100884 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822d177-19d7-490c-ab46-e0b426a78dad admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:19.130550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1112/2230] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:19.135201 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a822d177-19d7-490c-ab46-e0b426a78dad admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:19.139360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67709409-074f-4c80-b69c-37c8bcfb4a49 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:19.143336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-67709409-074f-4c80-b69c-37c8bcfb4a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:19.144435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1909db3-54ea-4e31-8aa2-6026b1b14258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:19.145396 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1909db3-54ea-4e31-8aa2-6026b1b14258 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:19.145754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1909db3-54ea-4e31-8aa2-6026b1b14258 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:19.146656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1909db3-54ea-4e31-8aa2-6026b1b14258 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:19.147610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1909db3-54ea-4e31-8aa2-6026b1b14258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:19.149784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1909db3-54ea-4e31-8aa2-6026b1b14258 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:19.166295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1119/2231] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:19.172831 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1909db3-54ea-4e31-8aa2-6026b1b14258 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:19.188113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1909db3-54ea-4e31-8aa2-6026b1b14258 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:19.194497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c7e4b-184d-4917-a91e-9b32e769f9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:19.195189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c7e4b-184d-4917-a91e-9b32e769f9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:19.195475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c7e4b-184d-4917-a91e-9b32e769f9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:19.195757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c7e4b-184d-4917-a91e-9b32e769f9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:19.197793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c7e4b-184d-4917-a91e-9b32e769f9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:19.200386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c7e4b-184d-4917-a91e-9b32e769f9f9 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:19.217610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1120/2232] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:19.219296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c7e4b-184d-4917-a91e-9b32e769f9f9 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:19.238137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c7e4b-184d-4917-a91e-9b32e769f9f9 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:19.243955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e993f-e680-430c-98c6-be0eddf3ccf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:19.244930 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e993f-e680-430c-98c6-be0eddf3ccf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:19.245280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e993f-e680-430c-98c6-be0eddf3ccf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:19.245589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e993f-e680-430c-98c6-be0eddf3ccf6 admin admin] PATH_INFO: `/v3/projects/abeac9f8541c46d99324d2cdd44cac74/users/e4d8f025bce5489c85b992405b93bd52/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:19.257905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67709409-074f-4c80-b69c-37c8bcfb4a49 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:19.270266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e993f-e680-430c-98c6-be0eddf3ccf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abeac9f8541c46d99324d2cdd44cac74, user_id=e4d8f025bce5489c85b992405b93bd52, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:19.274524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e993f-e680-430c-98c6-be0eddf3ccf6 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:19.277565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ebb15-9970-40ef-a2e2-2c0e756f578e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2b4bf2aa1744a46b6d2a1b5a136a0e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:19.278694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ebb15-9970-40ef-a2e2-2c0e756f578e service glance] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:19.281157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ebb15-9970-40ef-a2e2-2c0e756f578e service glance] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:19.282027 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ebb15-9970-40ef-a2e2-2c0e756f578e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:19.301921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1121/2233] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:54:19 2022] PUT /identity/v3/projects/abeac9f8541c46d99324d2cdd44cac74/users/e4d8f025bce5489c85b992405b93bd52/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:19.304256 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ebb15-9970-40ef-a2e2-2c0e756f578e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:19.305906 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ebb15-9970-40ef-a2e2-2c0e756f578e service glance] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:19.319164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e177f4-c61b-41da-9c99-edb6c992258a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:19.319514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e177f4-c61b-41da-9c99-edb6c992258a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:19.319900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e177f4-c61b-41da-9c99-edb6c992258a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:19.323347 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1113/2234] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:54:19.420821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-17e177f4-c61b-41da-9c99-edb6c992258a None None] MFA Rules not processed for user `e4d8f025bce5489c85b992405b93bd52`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:19.422405 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-17e177f4-c61b-41da-9c99-edb6c992258a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:19.508282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:19.576530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1122/2235] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:19.581142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ebb15-9970-40ef-a2e2-2c0e756f578e service glance] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:19.599540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ebb15-9970-40ef-a2e2-2c0e756f578e service glance] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:19.738453 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1c779-a303-41d8-873c-657d854b36db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:19.739514 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1c779-a303-41d8-873c-657d854b36db admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:19.739809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1c779-a303-41d8-873c-657d854b36db admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:19.740126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1c779-a303-41d8-873c-657d854b36db admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:19.740880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1c779-a303-41d8-873c-657d854b36db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:19.743115 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d1c779-a303-41d8-873c-657d854b36db admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:19.816469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e177f4-c61b-41da-9c99-edb6c992258a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:19.825443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1114/2236] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:54:19 2022] POST /identity/v3/projects => generated 385 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:19.835351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e177f4-c61b-41da-9c99-edb6c992258a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:19.957992 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4be18f-c9f8-41dd-8896-563fe1c329a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:19.958713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4be18f-c9f8-41dd-8896-563fe1c329a8 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:19.959004 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4be18f-c9f8-41dd-8896-563fe1c329a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:19.959297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4be18f-c9f8-41dd-8896-563fe1c329a8 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:19.960385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4be18f-c9f8-41dd-8896-563fe1c329a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:19.962732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4be18f-c9f8-41dd-8896-563fe1c329a8 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:20.011843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1c779-a303-41d8-873c-657d854b36db admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:20.014718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1123/2237] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:54:19 2022] POST /identity/v3/users => generated 299 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:20.035968 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1c779-a303-41d8-873c-657d854b36db admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:20.045501 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c8975-b598-4dd2-8a5c-1ca391caa01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:20.046485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c8975-b598-4dd2-8a5c-1ca391caa01f admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:20.046777 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c8975-b598-4dd2-8a5c-1ca391caa01f admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:20.047060 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c8975-b598-4dd2-8a5c-1ca391caa01f admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:20.047951 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c8975-b598-4dd2-8a5c-1ca391caa01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:20.059915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c8975-b598-4dd2-8a5c-1ca391caa01f admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:20.076758 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1115/2238] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:54:20.080898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4be18f-c9f8-41dd-8896-563fe1c329a8 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:20.099626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4be18f-c9f8-41dd-8896-563fe1c329a8 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:20.105544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ec031-a19b-4b80-890c-259c7be75490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:20.106626 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ec031-a19b-4b80-890c-259c7be75490 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:20.106964 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ec031-a19b-4b80-890c-259c7be75490 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:20.107341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ec031-a19b-4b80-890c-259c7be75490 admin admin] PATH_INFO: `/v3/projects/41749eae10524a83b1cb83cafa286cb7/users/a8dba5f966b84f0b81bd2e76f965f7be/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:20.178204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ec031-a19b-4b80-890c-259c7be75490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41749eae10524a83b1cb83cafa286cb7, user_id=a8dba5f966b84f0b81bd2e76f965f7be, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:20.182526 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ec031-a19b-4b80-890c-259c7be75490 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:20.211578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1124/2239] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:54:20 2022] PUT /identity/v3/projects/41749eae10524a83b1cb83cafa286cb7/users/a8dba5f966b84f0b81bd2e76f965f7be/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:20.215779 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c8975-b598-4dd2-8a5c-1ca391caa01f admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:20.228223 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c8975-b598-4dd2-8a5c-1ca391caa01f admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:20.279397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:20.299530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ec031-a19b-4b80-890c-259c7be75490 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:20.302448 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ae6ec031-a19b-4b80-890c-259c7be75490 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:20.360070 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39684f2d-11dd-40c8-8f79-92bd158f3f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:20.360816 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39684f2d-11dd-40c8-8f79-92bd158f3f95 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:20.361118 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39684f2d-11dd-40c8-8f79-92bd158f3f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:20.361465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39684f2d-11dd-40c8-8f79-92bd158f3f95 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:20.362340 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39684f2d-11dd-40c8-8f79-92bd158f3f95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:20.364296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39684f2d-11dd-40c8-8f79-92bd158f3f95 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:20.381997 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1116/2240] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:20.385949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39684f2d-11dd-40c8-8f79-92bd158f3f95 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:20.399431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39684f2d-11dd-40c8-8f79-92bd158f3f95 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:20.404493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8bd4a-af62-45a8-adff-97c1c5ed40c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:20.405479 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8bd4a-af62-45a8-adff-97c1c5ed40c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:20.405797 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8bd4a-af62-45a8-adff-97c1c5ed40c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:20.406150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ec031-a19b-4b80-890c-259c7be75490 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:20.406302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8bd4a-af62-45a8-adff-97c1c5ed40c2 admin admin] PATH_INFO: `/v3/projects/41749eae10524a83b1cb83cafa286cb7/users/a8dba5f966b84f0b81bd2e76f965f7be/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:20.408333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8bd4a-af62-45a8-adff-97c1c5ed40c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41749eae10524a83b1cb83cafa286cb7, user_id=a8dba5f966b84f0b81bd2e76f965f7be, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:20.411089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8bd4a-af62-45a8-adff-97c1c5ed40c2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:20.412456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8805c1-5f17-43ba-82c7-ed8c2abb65b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:20.413942 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8805c1-5f17-43ba-82c7-ed8c2abb65b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:20.414160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8805c1-5f17-43ba-82c7-ed8c2abb65b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:20.415618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8805c1-5f17-43ba-82c7-ed8c2abb65b6 admin admin] PATH_INFO: `/v3/users/be1201ac0aa7484f96e271410342a6b9` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:20.416878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8805c1-5f17-43ba-82c7-ed8c2abb65b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be1201ac0aa7484f96e271410342a6b9)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:20.419689 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8805c1-5f17-43ba-82c7-ed8c2abb65b6 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:20.428044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1117/2241] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:54:20 2022] PUT /identity/v3/projects/41749eae10524a83b1cb83cafa286cb7/users/a8dba5f966b84f0b81bd2e76f965f7be/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:20.435093 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31b00-d5d1-48ff-b1fb-ebab6c76400e None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:20.435671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31b00-d5d1-48ff-b1fb-ebab6c76400e None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:20.436211 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c31b00-d5d1-48ff-b1fb-ebab6c76400e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:20.475718 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-46c31b00-d5d1-48ff-b1fb-ebab6c76400e None None] MFA Rules not processed for user `a8dba5f966b84f0b81bd2e76f965f7be`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:20.477267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:20.477475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-46c31b00-d5d1-48ff-b1fb-ebab6c76400e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:20.486707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4e8805c1-5f17-43ba-82c7-ed8c2abb65b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be1201ac0aa7484f96e271410342a6b9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:20.506497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4e8805c1-5f17-43ba-82c7-ed8c2abb65b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be1201ac0aa7484f96e271410342a6b9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:20.518407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4e8805c1-5f17-43ba-82c7-ed8c2abb65b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be1201ac0aa7484f96e271410342a6b9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:20.519689 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4e8805c1-5f17-43ba-82c7-ed8c2abb65b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be1201ac0aa7484f96e271410342a6b9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:20.538253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-4e8805c1-5f17-43ba-82c7-ed8c2abb65b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be1201ac0aa7484f96e271410342a6b9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:20.545892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1125/2242] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:54:20 2022] DELETE /identity/v3/users/be1201ac0aa7484f96e271410342a6b9 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:20.596817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1118/2243] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:54:20.612739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8805c1-5f17-43ba-82c7-ed8c2abb65b6 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:20.614502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4e8805c1-5f17-43ba-82c7-ed8c2abb65b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:20.722671 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8805c1-5f17-43ba-82c7-ed8c2abb65b6 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:20.732186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc54a9d-1a6d-441f-bb9b-15bfbb118f68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:20.733330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc54a9d-1a6d-441f-bb9b-15bfbb118f68 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:20.733844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc54a9d-1a6d-441f-bb9b-15bfbb118f68 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:20.734355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc54a9d-1a6d-441f-bb9b-15bfbb118f68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:20.738651 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8bc54a9d-1a6d-441f-bb9b-15bfbb118f68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:20.851021 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc54a9d-1a6d-441f-bb9b-15bfbb118f68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:20.851558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc54a9d-1a6d-441f-bb9b-15bfbb118f68 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:20.871134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1126/2244] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:54:20.973874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c31b00-d5d1-48ff-b1fb-ebab6c76400e None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:20.975505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-46c31b00-d5d1-48ff-b1fb-ebab6c76400e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:21.104089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c31b00-d5d1-48ff-b1fb-ebab6c76400e None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:21.187627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:21.227173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656a535-d9ef-44e6-b671-b7ff66b7273a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:21.229074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656a535-d9ef-44e6-b671-b7ff66b7273a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:21.229371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656a535-d9ef-44e6-b671-b7ff66b7273a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:21.229824 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656a535-d9ef-44e6-b671-b7ff66b7273a admin admin] PATH_INFO: `/v3/projects/65ba80b0874640d289c54ad4ac5a1018` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:21.233544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656a535-d9ef-44e6-b671-b7ff66b7273a admin admin] RBAC: Authorizing `identity:delete_project(project_id=65ba80b0874640d289c54ad4ac5a1018)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:21.236405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656a535-d9ef-44e6-b671-b7ff66b7273a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:21.343102 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b656a535-d9ef-44e6-b671-b7ff66b7273a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65ba80b0874640d289c54ad4ac5a1018', 'request_id': 'req-b656a535-d9ef-44e6-b671-b7ff66b7273a'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:21.381201 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b656a535-d9ef-44e6-b671-b7ff66b7273a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65ba80b0874640d289c54ad4ac5a1018', 'request_id': 'req-b656a535-d9ef-44e6-b671-b7ff66b7273a'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:21.396342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b656a535-d9ef-44e6-b671-b7ff66b7273a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65ba80b0874640d289c54ad4ac5a1018', 'request_id': 'req-b656a535-d9ef-44e6-b671-b7ff66b7273a'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:21.418125 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b656a535-d9ef-44e6-b671-b7ff66b7273a admin admin] The token cache is being invalidate because project 65ba80b0874640d289c54ad4ac5a1018 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:54:21.418747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b656a535-d9ef-44e6-b671-b7ff66b7273a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:21.419437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b656a535-d9ef-44e6-b671-b7ff66b7273a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:21.424799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1119/2245] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:54:20 2022] DELETE /identity/v3/projects/65ba80b0874640d289c54ad4ac5a1018 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:21.427359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b3a7f-2449-4168-87b1-5f34d9bf0b65 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:21.427863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b3a7f-2449-4168-87b1-5f34d9bf0b65 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:21.428326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b3a7f-2449-4168-87b1-5f34d9bf0b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:21.458677 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-039b3a7f-2449-4168-87b1-5f34d9bf0b65 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:21.460228 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-039b3a7f-2449-4168-87b1-5f34d9bf0b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:21.594628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1127/2246] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:21.604432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323832df-8229-4b8e-8918-7c32ba1f7e0f None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:21.605006 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323832df-8229-4b8e-8918-7c32ba1f7e0f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:21.605814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323832df-8229-4b8e-8918-7c32ba1f7e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:21.631328 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-323832df-8229-4b8e-8918-7c32ba1f7e0f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:21.632814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-323832df-8229-4b8e-8918-7c32ba1f7e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:21.640977 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1120/2247] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:21.645320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b3a7f-2449-4168-87b1-5f34d9bf0b65 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:21.655578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b3a7f-2449-4168-87b1-5f34d9bf0b65 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:21.661342 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615bc552-32ba-4c28-95d9-9f4c30e25b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:21.662020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bc552-32ba-4c28-95d9-9f4c30e25b60 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:21.662310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bc552-32ba-4c28-95d9-9f4c30e25b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:21.662588 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615bc552-32ba-4c28-95d9-9f4c30e25b60 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:21.663345 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615bc552-32ba-4c28-95d9-9f4c30e25b60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:54:21.663663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615bc552-32ba-4c28-95d9-9f4c30e25b60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:21.665442 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615bc552-32ba-4c28-95d9-9f4c30e25b60 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:21.681438 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:21.693274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1128/2248] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:54:21.694469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323832df-8229-4b8e-8918-7c32ba1f7e0f None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:21.704245 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323832df-8229-4b8e-8918-7c32ba1f7e0f None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:21.709842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93991bf-8312-4c5c-bc52-ccf046274a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:21.710564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93991bf-8312-4c5c-bc52-ccf046274a14 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:21.710854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93991bf-8312-4c5c-bc52-ccf046274a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:21.711133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93991bf-8312-4c5c-bc52-ccf046274a14 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:21.711872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93991bf-8312-4c5c-bc52-ccf046274a14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:21.713807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93991bf-8312-4c5c-bc52-ccf046274a14 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:21.743240 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1121/2249] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:54:21 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:21.748349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615bc552-32ba-4c28-95d9-9f4c30e25b60 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:21.758966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615bc552-32ba-4c28-95d9-9f4c30e25b60 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:21.820815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cd9f3-9349-4ce2-8910-f367d72f561b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:21.821502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cd9f3-9349-4ce2-8910-f367d72f561b admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:21.821791 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cd9f3-9349-4ce2-8910-f367d72f561b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:21.822326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cd9f3-9349-4ce2-8910-f367d72f561b admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:21.823019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cd9f3-9349-4ce2-8910-f367d72f561b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:21.824976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cd9f3-9349-4ce2-8910-f367d72f561b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:21.852169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1129/2250] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:54:21 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:21.853376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93991bf-8312-4c5c-bc52-ccf046274a14 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:21.865071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93991bf-8312-4c5c-bc52-ccf046274a14 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:21.869898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95945bf-e4ae-473d-b534-f58d58b93bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:21.870566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95945bf-e4ae-473d-b534-f58d58b93bee admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:21.870854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95945bf-e4ae-473d-b534-f58d58b93bee admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:21.871133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95945bf-e4ae-473d-b534-f58d58b93bee admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:21.871869 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95945bf-e4ae-473d-b534-f58d58b93bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:21.873486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95945bf-e4ae-473d-b534-f58d58b93bee admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:21.883976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1122/2251] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:21.888391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cd9f3-9349-4ce2-8910-f367d72f561b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:21.899663 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cd9f3-9349-4ce2-8910-f367d72f561b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:21.904536 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4365149-b2d3-4dde-8d4e-9203c1efa922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:21.905230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4365149-b2d3-4dde-8d4e-9203c1efa922 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:21.905521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4365149-b2d3-4dde-8d4e-9203c1efa922 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:21.905803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4365149-b2d3-4dde-8d4e-9203c1efa922 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:21.906531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4365149-b2d3-4dde-8d4e-9203c1efa922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:21.908152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4365149-b2d3-4dde-8d4e-9203c1efa922 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:21.919116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1130/2252] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:21.923557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95945bf-e4ae-473d-b534-f58d58b93bee admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:21.934673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95945bf-e4ae-473d-b534-f58d58b93bee admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:21.939455 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51229e1f-e91b-4ab2-aa8e-a0b52663ed23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:21.940386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51229e1f-e91b-4ab2-aa8e-a0b52663ed23 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:21.940676 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51229e1f-e91b-4ab2-aa8e-a0b52663ed23 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:21.940949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51229e1f-e91b-4ab2-aa8e-a0b52663ed23 admin admin] PATH_INFO: `/v3/projects/8b1103c5cdec41ef85f6cacbe0ec7f86/users/1b9b8efaf17643e5b27f295e982162a5/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:21.955725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51229e1f-e91b-4ab2-aa8e-a0b52663ed23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b1103c5cdec41ef85f6cacbe0ec7f86, user_id=1b9b8efaf17643e5b27f295e982162a5, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:21.958913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51229e1f-e91b-4ab2-aa8e-a0b52663ed23 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:21.972665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1123/2253] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:54:21 2022] PUT /identity/v3/projects/8b1103c5cdec41ef85f6cacbe0ec7f86/users/1b9b8efaf17643e5b27f295e982162a5/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:54:21.982742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad39ce5-d5b1-4181-a6d4-046f2badf562 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:21.983040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad39ce5-d5b1-4181-a6d4-046f2badf562 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:21.983325 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad39ce5-d5b1-4181-a6d4-046f2badf562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:22.010551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-cad39ce5-d5b1-4181-a6d4-046f2badf562 None None] MFA Rules not processed for user `1b9b8efaf17643e5b27f295e982162a5`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:22.011851 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-cad39ce5-d5b1-4181-a6d4-046f2badf562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:22.098368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1131/2254] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:22.127265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51229e1f-e91b-4ab2-aa8e-a0b52663ed23 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:22.129672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-51229e1f-e91b-4ab2-aa8e-a0b52663ed23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:22.223424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51229e1f-e91b-4ab2-aa8e-a0b52663ed23 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:22.229343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708022ba-11f0-4388-b682-1b5982a9945e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:22.230002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708022ba-11f0-4388-b682-1b5982a9945e service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:22.230286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708022ba-11f0-4388-b682-1b5982a9945e service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:22.230562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708022ba-11f0-4388-b682-1b5982a9945e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:22.235474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:22.244347 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708022ba-11f0-4388-b682-1b5982a9945e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:22.246329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708022ba-11f0-4388-b682-1b5982a9945e service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:22.257881 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1124/2255] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:54:23.929939 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad39ce5-d5b1-4181-a6d4-046f2badf562 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:23.931368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-cad39ce5-d5b1-4181-a6d4-046f2badf562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:23.990467 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad39ce5-d5b1-4181-a6d4-046f2badf562 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:23.994493 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029893c-9b36-4229-b5d1-19465c20c194 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:23.995167 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029893c-9b36-4229-b5d1-19465c20c194 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:23.995478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029893c-9b36-4229-b5d1-19465c20c194 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:23.995758 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029893c-9b36-4229-b5d1-19465c20c194 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:23.997584 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3029893c-9b36-4229-b5d1-19465c20c194 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:24.017796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:24.058814 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029893c-9b36-4229-b5d1-19465c20c194 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:24.060284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029893c-9b36-4229-b5d1-19465c20c194 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:24.069536 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1132/2256] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:54:24.652824 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708022ba-11f0-4388-b682-1b5982a9945e service nova] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:24.654235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-708022ba-11f0-4388-b682-1b5982a9945e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:24.667206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708022ba-11f0-4388-b682-1b5982a9945e service nova] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:24.670765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2e300-74cc-4ddd-934a-e57b5c73b1dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:24.671409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2e300-74cc-4ddd-934a-e57b5c73b1dd service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:24.671694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2e300-74cc-4ddd-934a-e57b5c73b1dd service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:24.672005 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2e300-74cc-4ddd-934a-e57b5c73b1dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:24.673757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a5e2e300-74cc-4ddd-934a-e57b5c73b1dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:24.739966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2e300-74cc-4ddd-934a-e57b5c73b1dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:24.741335 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2e300-74cc-4ddd-934a-e57b5c73b1dd service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:24.750652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1125/2257] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:54:34.065389 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=18 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:34.747963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=33 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:37.129036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029893c-9b36-4229-b5d1-19465c20c194 service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:37.131254 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3029893c-9b36-4229-b5d1-19465c20c194 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:37.204944 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2e300-74cc-4ddd-934a-e57b5c73b1dd service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:37.222674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029893c-9b36-4229-b5d1-19465c20c194 service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:37.227442 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c411f0-3cc2-4b8e-a1d3-cc5c9924d7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:37.228357 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c411f0-3cc2-4b8e-a1d3-cc5c9924d7de admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:37.228645 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c411f0-3cc2-4b8e-a1d3-cc5c9924d7de admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:37.228929 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c411f0-3cc2-4b8e-a1d3-cc5c9924d7de admin admin] PATH_INFO: `/v3/users/6792a77be427458197c716454a3a8414` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:37.230154 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c411f0-3cc2-4b8e-a1d3-cc5c9924d7de admin admin] RBAC: Authorizing `identity:delete_user(user_id=6792a77be427458197c716454a3a8414)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:37.232134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c411f0-3cc2-4b8e-a1d3-cc5c9924d7de admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:37.270398 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2e300-74cc-4ddd-934a-e57b5c73b1dd service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:37.276005 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b84f27-8187-458d-9ae1-5acc4ee76b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:37.276985 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b84f27-8187-458d-9ae1-5acc4ee76b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:37.277543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b84f27-8187-458d-9ae1-5acc4ee76b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:37.278088 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b84f27-8187-458d-9ae1-5acc4ee76b82 admin admin] PATH_INFO: `/v3/users/1b9b8efaf17643e5b27f295e982162a5` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:37.279737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b84f27-8187-458d-9ae1-5acc4ee76b82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9b8efaf17643e5b27f295e982162a5)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:37.281688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b84f27-8187-458d-9ae1-5acc4ee76b82 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:37.292646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-92c411f0-3cc2-4b8e-a1d3-cc5c9924d7de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6792a77be427458197c716454a3a8414'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:37.305423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-92c411f0-3cc2-4b8e-a1d3-cc5c9924d7de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6792a77be427458197c716454a3a8414'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:37.313378 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-92c411f0-3cc2-4b8e-a1d3-cc5c9924d7de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6792a77be427458197c716454a3a8414'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:37.314078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-92c411f0-3cc2-4b8e-a1d3-cc5c9924d7de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6792a77be427458197c716454a3a8414'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:37.324937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-92c411f0-3cc2-4b8e-a1d3-cc5c9924d7de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6792a77be427458197c716454a3a8414'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:37.333062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1133/2258] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:54:37 2022] DELETE /identity/v3/users/6792a77be427458197c716454a3a8414 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:37.338361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe79e0e-c606-4a0c-a361-1fe44da51ab9 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:37.338831 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe79e0e-c606-4a0c-a361-1fe44da51ab9 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:37.339432 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe79e0e-c606-4a0c-a361-1fe44da51ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:37.341835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f5b84f27-8187-458d-9ae1-5acc4ee76b82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9b8efaf17643e5b27f295e982162a5'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:37.360122 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-4fe79e0e-c606-4a0c-a361-1fe44da51ab9 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:37.361597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4fe79e0e-c606-4a0c-a361-1fe44da51ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:37.363297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f5b84f27-8187-458d-9ae1-5acc4ee76b82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9b8efaf17643e5b27f295e982162a5'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:37.364279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f5b84f27-8187-458d-9ae1-5acc4ee76b82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9b8efaf17643e5b27f295e982162a5'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:37.387058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f5b84f27-8187-458d-9ae1-5acc4ee76b82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9b8efaf17643e5b27f295e982162a5'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:37.387745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f5b84f27-8187-458d-9ae1-5acc4ee76b82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9b8efaf17643e5b27f295e982162a5'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:37.408880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1126/2259] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:54:37 2022] DELETE /identity/v3/users/1b9b8efaf17643e5b27f295e982162a5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:54:37.414317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0d680-7975-4737-915c-3cc13806ff1f None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:37.414614 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0d680-7975-4737-915c-3cc13806ff1f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:37.415092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0d680-7975-4737-915c-3cc13806ff1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:37.443725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-e1c0d680-7975-4737-915c-3cc13806ff1f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:37.446814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e1c0d680-7975-4737-915c-3cc13806ff1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:37.450393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1134/2260] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:54:37.461044 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe79e0e-c606-4a0c-a361-1fe44da51ab9 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:37.462553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4fe79e0e-c606-4a0c-a361-1fe44da51ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:37.491748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:37.527304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1127/2261] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:54:37.533965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0d680-7975-4737-915c-3cc13806ff1f None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:37.538624 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe79e0e-c606-4a0c-a361-1fe44da51ab9 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:37.555019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:37.590329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e5315-a5b0-4d5a-a337-78b441fd48cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:37.590993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e5315-a5b0-4d5a-a337-78b441fd48cd service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:37.591283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e5315-a5b0-4d5a-a337-78b441fd48cd service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:37.591567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e5315-a5b0-4d5a-a337-78b441fd48cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:37.592718 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0d680-7975-4737-915c-3cc13806ff1f None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:37.596491 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d07c15-594b-466b-a5a2-9652345f4508 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:37.597672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d07c15-594b-466b-a5a2-9652345f4508 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:37.597967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d07c15-594b-466b-a5a2-9652345f4508 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:37.598228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d07c15-594b-466b-a5a2-9652345f4508 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:37.601799 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e5315-a5b0-4d5a-a337-78b441fd48cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:37.603246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662e5315-a5b0-4d5a-a337-78b441fd48cd service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:37.607435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d07c15-594b-466b-a5a2-9652345f4508 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:37.608905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d07c15-594b-466b-a5a2-9652345f4508 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:37.616329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1135/2262] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:54:37.622736 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1128/2263] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:54:38.207708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e5315-a5b0-4d5a-a337-78b441fd48cd service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:38.209169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-662e5315-a5b0-4d5a-a337-78b441fd48cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:38.212440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d07c15-594b-466b-a5a2-9652345f4508 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:38.214233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-29d07c15-594b-466b-a5a2-9652345f4508 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:38.239785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d07c15-594b-466b-a5a2-9652345f4508 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:38.244592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd5281-5c5a-4161-973d-91d0630f800d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:38.245470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd5281-5c5a-4161-973d-91d0630f800d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:38.245841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd5281-5c5a-4161-973d-91d0630f800d admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:38.246069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd5281-5c5a-4161-973d-91d0630f800d admin admin] PATH_INFO: `/v3/projects/8b1103c5cdec41ef85f6cacbe0ec7f86` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:38.247536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd5281-5c5a-4161-973d-91d0630f800d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b1103c5cdec41ef85f6cacbe0ec7f86)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:38.249535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd5281-5c5a-4161-973d-91d0630f800d admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:38.249903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e5315-a5b0-4d5a-a337-78b441fd48cd service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:38.255978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:38.257221 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:38.257515 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:38.257812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] PATH_INFO: `/v3/projects/af8b81470a69459fbd370a371672dfb4` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:38.259034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af8b81470a69459fbd370a371672dfb4)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:38.261025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:38.305397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-07fd5281-5c5a-4161-973d-91d0630f800d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b1103c5cdec41ef85f6cacbe0ec7f86', 'request_id': 'req-07fd5281-5c5a-4161-973d-91d0630f800d'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.313107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-07fd5281-5c5a-4161-973d-91d0630f800d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b1103c5cdec41ef85f6cacbe0ec7f86', 'request_id': 'req-07fd5281-5c5a-4161-973d-91d0630f800d'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.316153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af8b81470a69459fbd370a371672dfb4', 'request_id': 'req-0cc38284-4268-463c-97cf-c76075692e16'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.319729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-07fd5281-5c5a-4161-973d-91d0630f800d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b1103c5cdec41ef85f6cacbe0ec7f86', 'request_id': 'req-07fd5281-5c5a-4161-973d-91d0630f800d'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.325241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af8b81470a69459fbd370a371672dfb4', 'request_id': 'req-0cc38284-4268-463c-97cf-c76075692e16'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.333486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af8b81470a69459fbd370a371672dfb4', 'request_id': 'req-0cc38284-4268-463c-97cf-c76075692e16'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.334129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-07fd5281-5c5a-4161-973d-91d0630f800d admin admin] The token cache is being invalidate because project 8b1103c5cdec41ef85f6cacbe0ec7f86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:54:38.334458 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-07fd5281-5c5a-4161-973d-91d0630f800d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.335219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-07fd5281-5c5a-4161-973d-91d0630f800d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.339317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1129/2264] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:54:38 2022] DELETE /identity/v3/projects/8b1103c5cdec41ef85f6cacbe0ec7f86 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:54:38.342048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ab5bb-17ef-4022-bc2e-135152e8d397 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:38.343340 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ab5bb-17ef-4022-bc2e-135152e8d397 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:38.343340 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ab5bb-17ef-4022-bc2e-135152e8d397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:38.349106 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] The token cache is being invalidate because project af8b81470a69459fbd370a371672dfb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:54:38.349397 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.350089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.354531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1136/2265] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:54:38 2022] DELETE /identity/v3/projects/af8b81470a69459fbd370a371672dfb4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:38.355655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:38.357612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:38.362972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-901ab5bb-17ef-4022-bc2e-135152e8d397 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:38.364197 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-901ab5bb-17ef-4022-bc2e-135152e8d397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:38.425750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc38284-4268-463c-97cf-c76075692e16 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:38.430358 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37b996-c8c6-4952-95a1-fb10d451de37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:38.431229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1130/2266] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:54:38.431471 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37b996-c8c6-4952-95a1-fb10d451de37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:38.431581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37b996-c8c6-4952-95a1-fb10d451de37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:38.431774 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d37b996-c8c6-4952-95a1-fb10d451de37 admin admin] PATH_INFO: `/v3/users/e4d8f025bce5489c85b992405b93bd52` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:38.433007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37b996-c8c6-4952-95a1-fb10d451de37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4d8f025bce5489c85b992405b93bd52)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:38.434926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d37b996-c8c6-4952-95a1-fb10d451de37 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:38.437604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:38.440235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60f92a-5977-4404-9177-0b1a41745cfa None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:38.440738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60f92a-5977-4404-9177-0b1a41745cfa None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:38.441059 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60f92a-5977-4404-9177-0b1a41745cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:38.464236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5f60f92a-5977-4404-9177-0b1a41745cfa None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:38.465661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5f60f92a-5977-4404-9177-0b1a41745cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:38.475825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1131/2267] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:54:38.480185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60f92a-5977-4404-9177-0b1a41745cfa None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:38.489735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-9d37b996-c8c6-4952-95a1-fb10d451de37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4d8f025bce5489c85b992405b93bd52'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.492157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60f92a-5977-4404-9177-0b1a41745cfa None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:38.497127 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a04db-dc7f-459b-8f97-7f7a1fef8b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:38.497944 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a04db-dc7f-459b-8f97-7f7a1fef8b4c admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:38.498237 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a04db-dc7f-459b-8f97-7f7a1fef8b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:38.498515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a04db-dc7f-459b-8f97-7f7a1fef8b4c admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:38.499315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a04db-dc7f-459b-8f97-7f7a1fef8b4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:54:38.499629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a04db-dc7f-459b-8f97-7f7a1fef8b4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:38.501217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a04db-dc7f-459b-8f97-7f7a1fef8b4c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:38.502621 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-9d37b996-c8c6-4952-95a1-fb10d451de37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4d8f025bce5489c85b992405b93bd52'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.511631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-9d37b996-c8c6-4952-95a1-fb10d451de37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4d8f025bce5489c85b992405b93bd52'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.516280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-9d37b996-c8c6-4952-95a1-fb10d451de37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4d8f025bce5489c85b992405b93bd52'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.520016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1132/2268] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:54:38.524333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a04db-dc7f-459b-8f97-7f7a1fef8b4c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:38.530414 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-9d37b996-c8c6-4952-95a1-fb10d451de37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4d8f025bce5489c85b992405b93bd52'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:38.536420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a04db-dc7f-459b-8f97-7f7a1fef8b4c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:38.539498 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1137/2269] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:54:38 2022] DELETE /identity/v3/users/e4d8f025bce5489c85b992405b93bd52 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:54:38.541530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21a8c5-321a-411d-8b28-8cfc92fe021a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:38.542308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21a8c5-321a-411d-8b28-8cfc92fe021a admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:38.542701 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21a8c5-321a-411d-8b28-8cfc92fe021a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:38.543150 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21a8c5-321a-411d-8b28-8cfc92fe021a admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:38.544002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21a8c5-321a-411d-8b28-8cfc92fe021a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:38.546012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21a8c5-321a-411d-8b28-8cfc92fe021a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:38.577699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1133/2270] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:54:38 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:54:38.582620 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37b996-c8c6-4952-95a1-fb10d451de37 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:38.584253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9d37b996-c8c6-4952-95a1-fb10d451de37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:38.672550 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d37b996-c8c6-4952-95a1-fb10d451de37 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:38.745173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311417f-91f2-4691-8cdd-17f36f2fc2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:38.745861 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311417f-91f2-4691-8cdd-17f36f2fc2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:38.746144 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311417f-91f2-4691-8cdd-17f36f2fc2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:38.746422 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311417f-91f2-4691-8cdd-17f36f2fc2e3 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:38.747117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311417f-91f2-4691-8cdd-17f36f2fc2e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:38.749093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311417f-91f2-4691-8cdd-17f36f2fc2e3 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:38.770496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1138/2271] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:54:38 2022] POST /identity/v3/users => generated 282 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:38.775667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21a8c5-321a-411d-8b28-8cfc92fe021a admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:38.783780 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:38.791088 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21a8c5-321a-411d-8b28-8cfc92fe021a admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:38.796331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34782c66-6ddd-41a6-9739-bf56993a17ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:38.797018 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34782c66-6ddd-41a6-9739-bf56993a17ff admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:38.797509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34782c66-6ddd-41a6-9739-bf56993a17ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:38.797622 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34782c66-6ddd-41a6-9739-bf56993a17ff admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:38.798299 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34782c66-6ddd-41a6-9739-bf56993a17ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:38.799968 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34782c66-6ddd-41a6-9739-bf56993a17ff admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:38.812529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1134/2272] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:38.817003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311417f-91f2-4691-8cdd-17f36f2fc2e3 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:38.827737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311417f-91f2-4691-8cdd-17f36f2fc2e3 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:38.833327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc479ef8-5c21-4a01-b6bb-eee3fa18ed69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:38.834010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc479ef8-5c21-4a01-b6bb-eee3fa18ed69 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:38.834535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc479ef8-5c21-4a01-b6bb-eee3fa18ed69 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:38.834778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc479ef8-5c21-4a01-b6bb-eee3fa18ed69 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:38.835513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc479ef8-5c21-4a01-b6bb-eee3fa18ed69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:38.837161 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc479ef8-5c21-4a01-b6bb-eee3fa18ed69 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:38.850318 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1139/2273] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:38.854739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34782c66-6ddd-41a6-9739-bf56993a17ff admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:38.865971 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34782c66-6ddd-41a6-9739-bf56993a17ff admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:38.870355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adc2ef-a4da-4ea3-a6b8-d7f55d0c42e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:38.871343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adc2ef-a4da-4ea3-a6b8-d7f55d0c42e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:38.872387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adc2ef-a4da-4ea3-a6b8-d7f55d0c42e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:38.872873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6adc2ef-a4da-4ea3-a6b8-d7f55d0c42e6 admin admin] PATH_INFO: `/v3/projects/37dfbb9405f34a26bb8ba1fcd247c187/users/31c69c2efc924c999050cc3672308066/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:38.894393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6adc2ef-a4da-4ea3-a6b8-d7f55d0c42e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37dfbb9405f34a26bb8ba1fcd247c187, user_id=31c69c2efc924c999050cc3672308066, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:38.897101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6adc2ef-a4da-4ea3-a6b8-d7f55d0c42e6 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:38.911758 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1135/2274] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:54:38 2022] PUT /identity/v3/projects/37dfbb9405f34a26bb8ba1fcd247c187/users/31c69c2efc924c999050cc3672308066/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:38.923201 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad820f-1d90-426b-b73e-c27e7e46e191 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:38.923711 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad820f-1d90-426b-b73e-c27e7e46e191 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:38.924170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad820f-1d90-426b-b73e-c27e7e46e191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:38.937283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adc2ef-a4da-4ea3-a6b8-d7f55d0c42e6 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:38.938780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:38.939662 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f6adc2ef-a4da-4ea3-a6b8-d7f55d0c42e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:38.959992 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-6dad820f-1d90-426b-b73e-c27e7e46e191 None None] MFA Rules not processed for user `31c69c2efc924c999050cc3672308066`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:38.960695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6dad820f-1d90-426b-b73e-c27e7e46e191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:39.031745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6adc2ef-a4da-4ea3-a6b8-d7f55d0c42e6 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:39.037454 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:39.038577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:39.039029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:39.039593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] PATH_INFO: `/v3/projects/abeac9f8541c46d99324d2cdd44cac74` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:39.041649 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] RBAC: Authorizing `identity:delete_project(project_id=abeac9f8541c46d99324d2cdd44cac74)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:39.045036 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:39.046630 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1140/2275] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:38 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:39.051060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad820f-1d90-426b-b73e-c27e7e46e191 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:39.061742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad820f-1d90-426b-b73e-c27e7e46e191 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:39.067231 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf978c-a597-47cd-a541-d39ab0393e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:39.067958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf978c-a597-47cd-a541-d39ab0393e42 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:39.068216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf978c-a597-47cd-a541-d39ab0393e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:39.068501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcf978c-a597-47cd-a541-d39ab0393e42 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:39.069202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf978c-a597-47cd-a541-d39ab0393e42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:39.071160 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcf978c-a597-47cd-a541-d39ab0393e42 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:39.094334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abeac9f8541c46d99324d2cdd44cac74', 'request_id': 'req-b367986f-30ba-4c70-820b-49591b74b57c'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.102929 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abeac9f8541c46d99324d2cdd44cac74', 'request_id': 'req-b367986f-30ba-4c70-820b-49591b74b57c'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.103138 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1141/2276] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:54:39 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:54:39.106818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf978c-a597-47cd-a541-d39ab0393e42 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:39.109645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abeac9f8541c46d99324d2cdd44cac74', 'request_id': 'req-b367986f-30ba-4c70-820b-49591b74b57c'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.123035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] The token cache is being invalidate because project abeac9f8541c46d99324d2cdd44cac74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:54:39.123795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.124250 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.126037 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1136/2277] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:54:38 2022] DELETE /identity/v3/projects/abeac9f8541c46d99324d2cdd44cac74 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:54:39.130552 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:39.132478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:39.183241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcf978c-a597-47cd-a541-d39ab0393e42 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:39.188130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d2529-8786-4c61-8dc2-03d10882c66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:39.188809 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d2529-8786-4c61-8dc2-03d10882c66d admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:39.189099 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d2529-8786-4c61-8dc2-03d10882c66d admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:39.189389 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d2529-8786-4c61-8dc2-03d10882c66d admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:39.190073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d2529-8786-4c61-8dc2-03d10882c66d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:39.192785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d2529-8786-4c61-8dc2-03d10882c66d admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:39.200198 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367986f-30ba-4c70-820b-49591b74b57c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:39.205185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f8a22-89e0-43e3-8a18-21b2b04f2593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:39.206107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f8a22-89e0-43e3-8a18-21b2b04f2593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:39.207043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f8a22-89e0-43e3-8a18-21b2b04f2593 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:39.207335 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28f8a22-89e0-43e3-8a18-21b2b04f2593 admin admin] PATH_INFO: `/v3/users/a8dba5f966b84f0b81bd2e76f965f7be` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:39.208616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f8a22-89e0-43e3-8a18-21b2b04f2593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8dba5f966b84f0b81bd2e76f965f7be)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:39.210555 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28f8a22-89e0-43e3-8a18-21b2b04f2593 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:39.212513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:39.220963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1142/2278] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:54:39 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:54:39.222833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d2529-8786-4c61-8dc2-03d10882c66d admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:39.224914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-cf9d2529-8786-4c61-8dc2-03d10882c66d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:39.244064 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d2529-8786-4c61-8dc2-03d10882c66d admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:39.250093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b006e-940f-4a82-8dfb-0ae1a252cd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:39.251338 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b006e-940f-4a82-8dfb-0ae1a252cd3a admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:39.251631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b006e-940f-4a82-8dfb-0ae1a252cd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:39.252397 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66b006e-940f-4a82-8dfb-0ae1a252cd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:39.253048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b006e-940f-4a82-8dfb-0ae1a252cd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:39.255959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66b006e-940f-4a82-8dfb-0ae1a252cd3a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:39.271595 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1143/2279] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:39.273078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b006e-940f-4a82-8dfb-0ae1a252cd3a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:39.284200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66b006e-940f-4a82-8dfb-0ae1a252cd3a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:39.284743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d28f8a22-89e0-43e3-8a18-21b2b04f2593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8dba5f966b84f0b81bd2e76f965f7be'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.293085 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f361390-1982-4cdb-ae35-ed65604734bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:39.294221 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f361390-1982-4cdb-ae35-ed65604734bd admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:39.294574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f361390-1982-4cdb-ae35-ed65604734bd admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:39.294948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f361390-1982-4cdb-ae35-ed65604734bd admin admin] PATH_INFO: `/v3/projects/8bc7ab60b5d344b6a762bde55105e04a/users/e74606c6e6a44e6bb51d5fb0283ce4b9/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:39.305599 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d28f8a22-89e0-43e3-8a18-21b2b04f2593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8dba5f966b84f0b81bd2e76f965f7be'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.306369 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d28f8a22-89e0-43e3-8a18-21b2b04f2593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8dba5f966b84f0b81bd2e76f965f7be'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.314367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f361390-1982-4cdb-ae35-ed65604734bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bc7ab60b5d344b6a762bde55105e04a, user_id=e74606c6e6a44e6bb51d5fb0283ce4b9, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:39.315785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d28f8a22-89e0-43e3-8a18-21b2b04f2593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8dba5f966b84f0b81bd2e76f965f7be'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.316961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d28f8a22-89e0-43e3-8a18-21b2b04f2593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8dba5f966b84f0b81bd2e76f965f7be'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.317940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f361390-1982-4cdb-ae35-ed65604734bd admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:39.334669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1137/2280] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:54:39 2022] DELETE /identity/v3/users/a8dba5f966b84f0b81bd2e76f965f7be => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:54:39.335495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1144/2281] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:54:39 2022] PUT /identity/v3/projects/8bc7ab60b5d344b6a762bde55105e04a/users/e74606c6e6a44e6bb51d5fb0283ce4b9/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:39.340157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f8a22-89e0-43e3-8a18-21b2b04f2593 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:39.341832 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d28f8a22-89e0-43e3-8a18-21b2b04f2593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:39.427800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28f8a22-89e0-43e3-8a18-21b2b04f2593 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:39.434307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438eb90-ecb5-4652-8eae-810f62aab669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:39.435419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438eb90-ecb5-4652-8eae-810f62aab669 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:39.435419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438eb90-ecb5-4652-8eae-810f62aab669 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:39.435814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438eb90-ecb5-4652-8eae-810f62aab669 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:39.437017 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438eb90-ecb5-4652-8eae-810f62aab669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:39.439261 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438eb90-ecb5-4652-8eae-810f62aab669 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:39.457765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1138/2282] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:39.462344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f361390-1982-4cdb-ae35-ed65604734bd admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:39.513866 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:39.551574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f361390-1982-4cdb-ae35-ed65604734bd admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:39.557441 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92545a3-6e69-4941-bad5-276f77297034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:39.558567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92545a3-6e69-4941-bad5-276f77297034 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:39.558899 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92545a3-6e69-4941-bad5-276f77297034 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:39.559181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92545a3-6e69-4941-bad5-276f77297034 admin admin] PATH_INFO: `/v3/projects/8bc7ab60b5d344b6a762bde55105e04a/users/e74606c6e6a44e6bb51d5fb0283ce4b9/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:39.561234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92545a3-6e69-4941-bad5-276f77297034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bc7ab60b5d344b6a762bde55105e04a, user_id=e74606c6e6a44e6bb51d5fb0283ce4b9, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:39.563949 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92545a3-6e69-4941-bad5-276f77297034 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:39.584388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1145/2283] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:54:39 2022] PUT /identity/v3/projects/8bc7ab60b5d344b6a762bde55105e04a/users/e74606c6e6a44e6bb51d5fb0283ce4b9/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:39.586211 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91654b-d1ae-46e2-ba94-d48eecac4de3 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:39.586504 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91654b-d1ae-46e2-ba94-d48eecac4de3 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:39.586780 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd91654b-d1ae-46e2-ba94-d48eecac4de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:39.617775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-dd91654b-d1ae-46e2-ba94-d48eecac4de3 None None] MFA Rules not processed for user `e74606c6e6a44e6bb51d5fb0283ce4b9`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:39.618932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-dd91654b-d1ae-46e2-ba94-d48eecac4de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:39.641113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92545a3-6e69-4941-bad5-276f77297034 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:39.643384 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e92545a3-6e69-4941-bad5-276f77297034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:39.711545 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:39.718119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1139/2284] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:39 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:39.731564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91654b-d1ae-46e2-ba94-d48eecac4de3 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:39.733175 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-dd91654b-d1ae-46e2-ba94-d48eecac4de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:39.734481 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92545a3-6e69-4941-bad5-276f77297034 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:39.739816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:39.740727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:39.741139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:39.741851 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8 admin admin] PATH_INFO: `/v3/projects/41749eae10524a83b1cb83cafa286cb7` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:39.760945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41749eae10524a83b1cb83cafa286cb7)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:39.762886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:39.826037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41749eae10524a83b1cb83cafa286cb7', 'request_id': 'req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.827686 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd91654b-d1ae-46e2-ba94-d48eecac4de3 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:39.835805 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41749eae10524a83b1cb83cafa286cb7', 'request_id': 'req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.845187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41749eae10524a83b1cb83cafa286cb7', 'request_id': 'req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.859005 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8 admin admin] The token cache is being invalidate because project 41749eae10524a83b1cb83cafa286cb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:54:39.859300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.860032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-aa6ffed5-c975-4590-a213-b1bc7a4c05c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:54:39.861844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1146/2285] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:54:39 2022] DELETE /identity/v3/projects/41749eae10524a83b1cb83cafa286cb7 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:39.866479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caaa7e-a793-4d16-8dad-f10f45ff4ce0 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:39.866775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caaa7e-a793-4d16-8dad-f10f45ff4ce0 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:39.867067 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38caaa7e-a793-4d16-8dad-f10f45ff4ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:39.889015 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8aee6d-6777-4028-8e7d-9943158b64ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:39.889680 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8aee6d-6777-4028-8e7d-9943158b64ec service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:39.889967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8aee6d-6777-4028-8e7d-9943158b64ec service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:39.890852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8aee6d-6777-4028-8e7d-9943158b64ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:39.891209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-38caaa7e-a793-4d16-8dad-f10f45ff4ce0 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:39.892584 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-38caaa7e-a793-4d16-8dad-f10f45ff4ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:39.892767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-fe8aee6d-6777-4028-8e7d-9943158b64ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:39.952390 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:39.972710 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1147/2286] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:54:39.973594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8aee6d-6777-4028-8e7d-9943158b64ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:39.975056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8aee6d-6777-4028-8e7d-9943158b64ec service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:39.980689 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558288b-6e13-4190-924d-1b9b24e0eda5 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:39.981252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558288b-6e13-4190-924d-1b9b24e0eda5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:39.981659 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558288b-6e13-4190-924d-1b9b24e0eda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:39.985654 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1140/2287] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:39 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:54:40.004157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-5558288b-6e13-4190-924d-1b9b24e0eda5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:40.005376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5558288b-6e13-4190-924d-1b9b24e0eda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:40.016654 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1148/2288] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:54:40.022412 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8aee6d-6777-4028-8e7d-9943158b64ec service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:40.045192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8aee6d-6777-4028-8e7d-9943158b64ec service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:40.067581 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f270c3d-554d-4c57-8041-424c936bec2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:40.068778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f270c3d-554d-4c57-8041-424c936bec2b admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:40.069229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f270c3d-554d-4c57-8041-424c936bec2b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:40.069692 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f270c3d-554d-4c57-8041-424c936bec2b admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:40.070978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f270c3d-554d-4c57-8041-424c936bec2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:54:40.071496 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f270c3d-554d-4c57-8041-424c936bec2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:40.076578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f270c3d-554d-4c57-8041-424c936bec2b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:40.114574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558288b-6e13-4190-924d-1b9b24e0eda5 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:40.116607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5558288b-6e13-4190-924d-1b9b24e0eda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:40.120727 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1141/2289] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:54:40.123650 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f270c3d-554d-4c57-8041-424c936bec2b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:40.138140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f270c3d-554d-4c57-8041-424c936bec2b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:40.142990 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558288b-6e13-4190-924d-1b9b24e0eda5 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:40.143325 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef01cb-ecef-403d-a20d-1c850ec22e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:40.144066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef01cb-ecef-403d-a20d-1c850ec22e6c admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:40.144358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef01cb-ecef-403d-a20d-1c850ec22e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:40.144644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef01cb-ecef-403d-a20d-1c850ec22e6c admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:40.145351 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef01cb-ecef-403d-a20d-1c850ec22e6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:40.147423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef01cb-ecef-403d-a20d-1c850ec22e6c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:40.148481 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee78235-4fe2-4210-a2f1-459e9d0c34ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:40.149369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee78235-4fe2-4210-a2f1-459e9d0c34ef service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:40.149881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee78235-4fe2-4210-a2f1-459e9d0c34ef service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:40.150351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee78235-4fe2-4210-a2f1-459e9d0c34ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:40.152453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0ee78235-4fe2-4210-a2f1-459e9d0c34ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:40.186052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1142/2290] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:54:40 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:40.189913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef01cb-ecef-403d-a20d-1c850ec22e6c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:40.242182 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee78235-4fe2-4210-a2f1-459e9d0c34ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:40.243244 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:40.244056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee78235-4fe2-4210-a2f1-459e9d0c34ef service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:40.259586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1149/2291] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:40 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:54:40.313342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef01cb-ecef-403d-a20d-1c850ec22e6c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:40.319769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38261399-67ee-4a5f-91b5-e25bd6505e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:40.320972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38261399-67ee-4a5f-91b5-e25bd6505e5c admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:40.321932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38261399-67ee-4a5f-91b5-e25bd6505e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:40.322296 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38261399-67ee-4a5f-91b5-e25bd6505e5c admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:40.329862 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38261399-67ee-4a5f-91b5-e25bd6505e5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:40.332988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38261399-67ee-4a5f-91b5-e25bd6505e5c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:40.361149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1143/2292] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:54:40 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:54:40.366202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee78235-4fe2-4210-a2f1-459e9d0c34ef service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:40.382983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee78235-4fe2-4210-a2f1-459e9d0c34ef service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:40.388171 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e29cb-2288-41bf-b6c5-622df538f34f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:40.389243 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e29cb-2288-41bf-b6c5-622df538f34f admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:40.389243 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e29cb-2288-41bf-b6c5-622df538f34f admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:40.389463 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549e29cb-2288-41bf-b6c5-622df538f34f admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:40.390144 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e29cb-2288-41bf-b6c5-622df538f34f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:40.391950 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549e29cb-2288-41bf-b6c5-622df538f34f admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:40.406913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1150/2293] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:54:40.408052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38261399-67ee-4a5f-91b5-e25bd6505e5c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:40.422429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38261399-67ee-4a5f-91b5-e25bd6505e5c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:40.428842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1fb9d-96e7-45b0-a790-303dd614dbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:40.429665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1fb9d-96e7-45b0-a790-303dd614dbcf admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:40.430016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1fb9d-96e7-45b0-a790-303dd614dbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:40.430374 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1fb9d-96e7-45b0-a790-303dd614dbcf admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:40.431203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1fb9d-96e7-45b0-a790-303dd614dbcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:40.432941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d1fb9d-96e7-45b0-a790-303dd614dbcf admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:40.463490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1144/2294] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:40 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:54:40.464893 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e29cb-2288-41bf-b6c5-622df538f34f admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:40.478463 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549e29cb-2288-41bf-b6c5-622df538f34f admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:40.486653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee6f79-72e2-49eb-9d88-8b63341c8e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:40.488452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee6f79-72e2-49eb-9d88-8b63341c8e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:40.488856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee6f79-72e2-49eb-9d88-8b63341c8e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:40.489391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ee6f79-72e2-49eb-9d88-8b63341c8e6f admin admin] PATH_INFO: `/v3/projects/96004c35e9a84f2ab6b0601e28f45627/users/5d24ace76e3e4f10b63a53bb7482ac07/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:40.506836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee6f79-72e2-49eb-9d88-8b63341c8e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=96004c35e9a84f2ab6b0601e28f45627, user_id=5d24ace76e3e4f10b63a53bb7482ac07, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:40.511643 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ee6f79-72e2-49eb-9d88-8b63341c8e6f admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:40.523947 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1151/2295] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:54:40 2022] PUT /identity/v3/projects/96004c35e9a84f2ab6b0601e28f45627/users/5d24ace76e3e4f10b63a53bb7482ac07/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:40.528831 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf993145-b658-433c-95f9-0ab74bc35567 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:40.529118 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf993145-b658-433c-95f9-0ab74bc35567 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:40.529395 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf993145-b658-433c-95f9-0ab74bc35567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:40.547967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-cf993145-b658-433c-95f9-0ab74bc35567 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:40.549231 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-cf993145-b658-433c-95f9-0ab74bc35567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:40.611204 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1145/2296] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:54:40.621618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee6f79-72e2-49eb-9d88-8b63341c8e6f admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:40.623395 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-91ee6f79-72e2-49eb-9d88-8b63341c8e6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:40.682391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ee6f79-72e2-49eb-9d88-8b63341c8e6f admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:40.686237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e6dab-5f2e-4d13-ab12-229c252831c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:40.686884 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e6dab-5f2e-4d13-ab12-229c252831c9 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:40.687167 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e6dab-5f2e-4d13-ab12-229c252831c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:40.687442 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e6dab-5f2e-4d13-ab12-229c252831c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:40.696704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e6dab-5f2e-4d13-ab12-229c252831c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:40.698146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e6dab-5f2e-4d13-ab12-229c252831c9 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:40.708025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1152/2297] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:54:45.358783 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c240fb2-9b7d-4c21-92ae-b8f05bf5bd9d None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:45.359288 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c240fb2-9b7d-4c21-92ae-b8f05bf5bd9d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:45.359368 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c240fb2-9b7d-4c21-92ae-b8f05bf5bd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:45.379685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-3c240fb2-9b7d-4c21-92ae-b8f05bf5bd9d None None] MFA Rules not processed for user `d3b3a526943646dcba7bb1806601696b`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:45.380965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3c240fb2-9b7d-4c21-92ae-b8f05bf5bd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:45.512848 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1146/2298] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:46.765470 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afcf1f-2210-49fb-93c6-6ce89aa3f367 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:46.766367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afcf1f-2210-49fb-93c6-6ce89aa3f367 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:46.767265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86afcf1f-2210-49fb-93c6-6ce89aa3f367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:46.814405 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-86afcf1f-2210-49fb-93c6-6ce89aa3f367 None None] MFA Rules not processed for user `5d24ace76e3e4f10b63a53bb7482ac07`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:46.816438 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-86afcf1f-2210-49fb-93c6-6ce89aa3f367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:46.881535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:46.911268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1153/2299] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:46.915755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c240fb2-9b7d-4c21-92ae-b8f05bf5bd9d None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:46.927593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c240fb2-9b7d-4c21-92ae-b8f05bf5bd9d None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:46.932083 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836feb0-ec94-4f21-93a6-bdb4057e38b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:46.932744 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836feb0-ec94-4f21-93a6-bdb4057e38b3 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:46.933030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836feb0-ec94-4f21-93a6-bdb4057e38b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:46.933304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2836feb0-ec94-4f21-93a6-bdb4057e38b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:46.934001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836feb0-ec94-4f21-93a6-bdb4057e38b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:46.935909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2836feb0-ec94-4f21-93a6-bdb4057e38b3 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:46.968009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1147/2300] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:54:46 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:46.971739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afcf1f-2210-49fb-93c6-6ce89aa3f367 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:46.983467 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86afcf1f-2210-49fb-93c6-6ce89aa3f367 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:47.042134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd931a40-cda5-4389-bfbd-a7d0339f098a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:47.042812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd931a40-cda5-4389-bfbd-a7d0339f098a admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:47.043116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd931a40-cda5-4389-bfbd-a7d0339f098a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:47.043437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd931a40-cda5-4389-bfbd-a7d0339f098a admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:47.044252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd931a40-cda5-4389-bfbd-a7d0339f098a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:47.046189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd931a40-cda5-4389-bfbd-a7d0339f098a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:47.075151 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1154/2301] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:54:46 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:54:47.078756 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836feb0-ec94-4f21-93a6-bdb4057e38b3 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:47.089537 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2836feb0-ec94-4f21-93a6-bdb4057e38b3 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:47.094383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b39fc-27a4-4f66-a407-ee15f44183ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:47.095063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b39fc-27a4-4f66-a407-ee15f44183ed admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:47.095342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b39fc-27a4-4f66-a407-ee15f44183ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:47.095616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b39fc-27a4-4f66-a407-ee15f44183ed admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:47.096789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b39fc-27a4-4f66-a407-ee15f44183ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:47.098398 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b39fc-27a4-4f66-a407-ee15f44183ed admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:47.111003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1148/2302] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:54:47.115742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd931a40-cda5-4389-bfbd-a7d0339f098a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:47.126247 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd931a40-cda5-4389-bfbd-a7d0339f098a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:47.131608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765d65c-d2c9-4b83-a37d-78079d75c0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:47.132580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765d65c-d2c9-4b83-a37d-78079d75c0b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:47.132869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765d65c-d2c9-4b83-a37d-78079d75c0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:47.133149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765d65c-d2c9-4b83-a37d-78079d75c0b0 admin admin] PATH_INFO: `/v3/projects/4108e5427f894fd39fc682d56b455b8f/users/c4e1e194e6dc4a3dafbba25de07d6e43/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:47.147984 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765d65c-d2c9-4b83-a37d-78079d75c0b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4108e5427f894fd39fc682d56b455b8f, user_id=c4e1e194e6dc4a3dafbba25de07d6e43, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:47.150627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765d65c-d2c9-4b83-a37d-78079d75c0b0 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:47.164942 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1155/2303] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:54:47 2022] PUT /identity/v3/projects/4108e5427f894fd39fc682d56b455b8f/users/c4e1e194e6dc4a3dafbba25de07d6e43/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:47.168967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b39fc-27a4-4f66-a407-ee15f44183ed admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:47.180420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b39fc-27a4-4f66-a407-ee15f44183ed admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:47.202283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:47.244324 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b0fc8-67bd-40da-891b-e176d4c8f45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:47.244997 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b0fc8-67bd-40da-891b-e176d4c8f45b admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:47.245278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b0fc8-67bd-40da-891b-e176d4c8f45b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:47.245550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b0fc8-67bd-40da-891b-e176d4c8f45b admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:47.246278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b0fc8-67bd-40da-891b-e176d4c8f45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:47.247860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b0fc8-67bd-40da-891b-e176d4c8f45b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:47.257651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1149/2304] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:54:47.261744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765d65c-d2c9-4b83-a37d-78079d75c0b0 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:47.273558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765d65c-d2c9-4b83-a37d-78079d75c0b0 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:47.279949 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df7ece-3d22-4b8d-9e96-87388769f67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:47.280876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df7ece-3d22-4b8d-9e96-87388769f67f admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:47.281157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df7ece-3d22-4b8d-9e96-87388769f67f admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:47.281459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df7ece-3d22-4b8d-9e96-87388769f67f admin admin] PATH_INFO: `/v3/projects/4108e5427f894fd39fc682d56b455b8f/users/c4e1e194e6dc4a3dafbba25de07d6e43/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:47.283500 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df7ece-3d22-4b8d-9e96-87388769f67f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4108e5427f894fd39fc682d56b455b8f, user_id=c4e1e194e6dc4a3dafbba25de07d6e43, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:47.286709 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df7ece-3d22-4b8d-9e96-87388769f67f admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:47.297974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1156/2305] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:54:47 2022] PUT /identity/v3/projects/4108e5427f894fd39fc682d56b455b8f/users/c4e1e194e6dc4a3dafbba25de07d6e43/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:54:52.257447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fcb7e-fc18-4df1-bcdc-d8af6069fa3a None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:52.257997 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fcb7e-fc18-4df1-bcdc-d8af6069fa3a None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:52.257997 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fcb7e-fc18-4df1-bcdc-d8af6069fa3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:52.290698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-e26fcb7e-fc18-4df1-bcdc-d8af6069fa3a None None] MFA Rules not processed for user `c4e1e194e6dc4a3dafbba25de07d6e43`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:52.291962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-e26fcb7e-fc18-4df1-bcdc-d8af6069fa3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:52.376651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1150/2306] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:52 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:54:52.391391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df7ece-3d22-4b8d-9e96-87388769f67f admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:54:52.393856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-82df7ece-3d22-4b8d-9e96-87388769f67f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:52.477935 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df7ece-3d22-4b8d-9e96-87388769f67f admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:54:52.484195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff3dc1-b7bd-4794-99a7-16bcaf006f8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:54:52.484874 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff3dc1-b7bd-4794-99a7-16bcaf006f8f service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:52.485164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff3dc1-b7bd-4794-99a7-16bcaf006f8f service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:52.485439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff3dc1-b7bd-4794-99a7-16bcaf006f8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:52.519101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:54:52.548531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff3dc1-b7bd-4794-99a7-16bcaf006f8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:54:52.552126 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff3dc1-b7bd-4794-99a7-16bcaf006f8f service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:54:52.561698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1157/2307] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:54:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:54:53.719499 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe1232-2f4d-4b6b-9fcb-e41e3bc90dc4 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:54:53.720835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe1232-2f4d-4b6b-9fcb-e41e3bc90dc4 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:54:53.721391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe1232-2f4d-4b6b-9fcb-e41e3bc90dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:54:53.757292 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5dbe1232-2f4d-4b6b-9fcb-e41e3bc90dc4 None None] MFA Rules not processed for user `5d24ace76e3e4f10b63a53bb7482ac07`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:54:53.758941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5dbe1232-2f4d-4b6b-9fcb-e41e3bc90dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:54:53.769670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1151/2308] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:54:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:01.346924 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff3dc1-b7bd-4794-99a7-16bcaf006f8f service nova] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:01.348484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0fff3dc1-b7bd-4794-99a7-16bcaf006f8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:01.436700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff3dc1-b7bd-4794-99a7-16bcaf006f8f service nova] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:01.444463 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d998c-1bc8-45c6-ba86-6e1a0eaa0fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:01.445956 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d998c-1bc8-45c6-ba86-6e1a0eaa0fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:01.446443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d998c-1bc8-45c6-ba86-6e1a0eaa0fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:01.446957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8d998c-1bc8-45c6-ba86-6e1a0eaa0fe1 admin admin] PATH_INFO: `/v3/users/31c69c2efc924c999050cc3672308066` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:01.449173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d998c-1bc8-45c6-ba86-6e1a0eaa0fe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c69c2efc924c999050cc3672308066)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:01.453320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8d998c-1bc8-45c6-ba86-6e1a0eaa0fe1 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:01.532480 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-bf8d998c-1bc8-45c6-ba86-6e1a0eaa0fe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c69c2efc924c999050cc3672308066'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:01.547438 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-bf8d998c-1bc8-45c6-ba86-6e1a0eaa0fe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c69c2efc924c999050cc3672308066'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:01.556387 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-bf8d998c-1bc8-45c6-ba86-6e1a0eaa0fe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c69c2efc924c999050cc3672308066'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:01.557040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-bf8d998c-1bc8-45c6-ba86-6e1a0eaa0fe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c69c2efc924c999050cc3672308066'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:01.565008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-bf8d998c-1bc8-45c6-ba86-6e1a0eaa0fe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c69c2efc924c999050cc3672308066'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:01.571630 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1158/2309] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:55:01 2022] DELETE /identity/v3/users/31c69c2efc924c999050cc3672308066 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:01.577305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964a97f-3b39-4a7a-bb8f-5dc2f97b6d85 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:01.577914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964a97f-3b39-4a7a-bb8f-5dc2f97b6d85 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:01.578028 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8964a97f-3b39-4a7a-bb8f-5dc2f97b6d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:01.599813 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-8964a97f-3b39-4a7a-bb8f-5dc2f97b6d85 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:01.601104 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-8964a97f-3b39-4a7a-bb8f-5dc2f97b6d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:01.683607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1152/2310] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:55:01.696623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d998c-1bc8-45c6-ba86-6e1a0eaa0fe1 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:01.698494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-bf8d998c-1bc8-45c6-ba86-6e1a0eaa0fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:01.849518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8d998c-1bc8-45c6-ba86-6e1a0eaa0fe1 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:01.856794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd2112-85b1-4367-99b9-61b7fce7ad05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:01.857971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2112-85b1-4367-99b9-61b7fce7ad05 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:01.858473 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2112-85b1-4367-99b9-61b7fce7ad05 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:01.858967 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd2112-85b1-4367-99b9-61b7fce7ad05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:01.870685 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:01.877563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd2112-85b1-4367-99b9-61b7fce7ad05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:01.879304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bd2112-85b1-4367-99b9-61b7fce7ad05 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:01.892747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1159/2311] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:55:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:55:02.249879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964a97f-3b39-4a7a-bb8f-5dc2f97b6d85 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:02.251707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-8964a97f-3b39-4a7a-bb8f-5dc2f97b6d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:02.272441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8964a97f-3b39-4a7a-bb8f-5dc2f97b6d85 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:02.277909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73b9637-62d8-4297-981a-989c9a03f0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:02.279723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b9637-62d8-4297-981a-989c9a03f0d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:02.279723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b9637-62d8-4297-981a-989c9a03f0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:02.279723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73b9637-62d8-4297-981a-989c9a03f0d6 admin admin] PATH_INFO: `/v3/users/2a5727624fa84f1da109dbe438ef750e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:02.280974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73b9637-62d8-4297-981a-989c9a03f0d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a5727624fa84f1da109dbe438ef750e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:02.283309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73b9637-62d8-4297-981a-989c9a03f0d6 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:02.309481 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd2112-85b1-4367-99b9-61b7fce7ad05 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:02.312628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d5bd2112-85b1-4367-99b9-61b7fce7ad05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:02.341470 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd2112-85b1-4367-99b9-61b7fce7ad05 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:02.348904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:02.350025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:02.350196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:02.350362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] PATH_INFO: `/v3/projects/37dfbb9405f34a26bb8ba1fcd247c187` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:02.351269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:02.351683 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37dfbb9405f34a26bb8ba1fcd247c187)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:02.354320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:02.402512 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e73b9637-62d8-4297-981a-989c9a03f0d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a5727624fa84f1da109dbe438ef750e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.435469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e73b9637-62d8-4297-981a-989c9a03f0d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a5727624fa84f1da109dbe438ef750e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.437470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e73b9637-62d8-4297-981a-989c9a03f0d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a5727624fa84f1da109dbe438ef750e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.450642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37dfbb9405f34a26bb8ba1fcd247c187', 'request_id': 'req-7ce18018-15ab-4bec-9f63-0b1f5d46e357'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.454012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e73b9637-62d8-4297-981a-989c9a03f0d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a5727624fa84f1da109dbe438ef750e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.454892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e73b9637-62d8-4297-981a-989c9a03f0d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a5727624fa84f1da109dbe438ef750e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.475245 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37dfbb9405f34a26bb8ba1fcd247c187', 'request_id': 'req-7ce18018-15ab-4bec-9f63-0b1f5d46e357'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.477168 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1153/2312] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:55:02 2022] DELETE /identity/v3/users/2a5727624fa84f1da109dbe438ef750e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:55:02.482604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a24ac4-ba51-4219-b4e5-afa12b5d9492 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:02.482888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a24ac4-ba51-4219-b4e5-afa12b5d9492 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:02.483183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a24ac4-ba51-4219-b4e5-afa12b5d9492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:02.488572 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37dfbb9405f34a26bb8ba1fcd247c187', 'request_id': 'req-7ce18018-15ab-4bec-9f63-0b1f5d46e357'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.505571 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-59a24ac4-ba51-4219-b4e5-afa12b5d9492 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:02.506835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-59a24ac4-ba51-4219-b4e5-afa12b5d9492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:02.515911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] The token cache is being invalidate because project 37dfbb9405f34a26bb8ba1fcd247c187 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:55:02.516787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.518143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.521355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1160/2313] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:55:02 2022] DELETE /identity/v3/projects/37dfbb9405f34a26bb8ba1fcd247c187 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:55:02.526617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:02.529048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:02.608539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1154/2314] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:02.622695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a24ac4-ba51-4219-b4e5-afa12b5d9492 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:02.625025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-59a24ac4-ba51-4219-b4e5-afa12b5d9492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:02.627811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce18018-15ab-4bec-9f63-0b1f5d46e357 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:02.633719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c6190-d578-45b1-b207-4696ca042b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:02.635368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c6190-d578-45b1-b207-4696ca042b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:02.635843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c6190-d578-45b1-b207-4696ca042b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:02.636499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c6190-d578-45b1-b207-4696ca042b3b admin admin] PATH_INFO: `/v3/users/e74606c6e6a44e6bb51d5fb0283ce4b9` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:02.638421 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c6190-d578-45b1-b207-4696ca042b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e74606c6e6a44e6bb51d5fb0283ce4b9)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:02.641532 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c6190-d578-45b1-b207-4696ca042b3b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:02.707663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a24ac4-ba51-4219-b4e5-afa12b5d9492 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:02.715178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e67fd-6c8d-404b-abfe-cdd8d761a878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:02.716315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e67fd-6c8d-404b-abfe-cdd8d761a878 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:02.716825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e67fd-6c8d-404b-abfe-cdd8d761a878 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:02.717504 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e67fd-6c8d-404b-abfe-cdd8d761a878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:02.722519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7f9c6190-d578-45b1-b207-4696ca042b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e74606c6e6a44e6bb51d5fb0283ce4b9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.736658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e67fd-6c8d-404b-abfe-cdd8d761a878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:02.737247 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e67fd-6c8d-404b-abfe-cdd8d761a878 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:02.746728 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7f9c6190-d578-45b1-b207-4696ca042b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e74606c6e6a44e6bb51d5fb0283ce4b9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.749675 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1155/2315] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:55:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:55:02.772079 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7f9c6190-d578-45b1-b207-4696ca042b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e74606c6e6a44e6bb51d5fb0283ce4b9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.773344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7f9c6190-d578-45b1-b207-4696ca042b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e74606c6e6a44e6bb51d5fb0283ce4b9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.787068 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:02.791835 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7f9c6190-d578-45b1-b207-4696ca042b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e74606c6e6a44e6bb51d5fb0283ce4b9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:02.808930 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1161/2316] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:55:02 2022] DELETE /identity/v3/users/e74606c6e6a44e6bb51d5fb0283ce4b9 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:55:03.523824 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e67fd-6c8d-404b-abfe-cdd8d761a878 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:03.527306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3c6e67fd-6c8d-404b-abfe-cdd8d761a878 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:03.554269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e67fd-6c8d-404b-abfe-cdd8d761a878 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:03.585170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c6190-d578-45b1-b207-4696ca042b3b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:03.592470 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7f9c6190-d578-45b1-b207-4696ca042b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:03.722860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:03.745411 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c6190-d578-45b1-b207-4696ca042b3b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:03.751365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97ecb7-e261-446c-a9f0-869bbfc12a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:03.752352 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97ecb7-e261-446c-a9f0-869bbfc12a27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:03.752766 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97ecb7-e261-446c-a9f0-869bbfc12a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:03.752937 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97ecb7-e261-446c-a9f0-869bbfc12a27 admin admin] PATH_INFO: `/v3/projects/8bc7ab60b5d344b6a762bde55105e04a` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:03.753553 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f3e11-6960-4c80-96c2-bcc23af6ee56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:03.754248 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97ecb7-e261-446c-a9f0-869bbfc12a27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bc7ab60b5d344b6a762bde55105e04a)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:03.755498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f3e11-6960-4c80-96c2-bcc23af6ee56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:03.756146 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f3e11-6960-4c80-96c2-bcc23af6ee56 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:03.756301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97ecb7-e261-446c-a9f0-869bbfc12a27 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:03.756871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f3e11-6960-4c80-96c2-bcc23af6ee56 admin admin] PATH_INFO: `/v3/projects/f329efa85eee4255b208df73dafb3732` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:03.759336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f3e11-6960-4c80-96c2-bcc23af6ee56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f329efa85eee4255b208df73dafb3732)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:03.763678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f3e11-6960-4c80-96c2-bcc23af6ee56 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:03.823463 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1f97ecb7-e261-446c-a9f0-869bbfc12a27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bc7ab60b5d344b6a762bde55105e04a', 'request_id': 'req-1f97ecb7-e261-446c-a9f0-869bbfc12a27'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:03.840494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1f97ecb7-e261-446c-a9f0-869bbfc12a27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bc7ab60b5d344b6a762bde55105e04a', 'request_id': 'req-1f97ecb7-e261-446c-a9f0-869bbfc12a27'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:03.851454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1f97ecb7-e261-446c-a9f0-869bbfc12a27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bc7ab60b5d344b6a762bde55105e04a', 'request_id': 'req-1f97ecb7-e261-446c-a9f0-869bbfc12a27'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:03.869773 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-052f3e11-6960-4c80-96c2-bcc23af6ee56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f329efa85eee4255b208df73dafb3732', 'request_id': 'req-052f3e11-6960-4c80-96c2-bcc23af6ee56'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:03.876955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1f97ecb7-e261-446c-a9f0-869bbfc12a27 admin admin] The token cache is being invalidate because project 8bc7ab60b5d344b6a762bde55105e04a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:55:03.876955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1f97ecb7-e261-446c-a9f0-869bbfc12a27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:03.877526 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1f97ecb7-e261-446c-a9f0-869bbfc12a27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:03.886023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1162/2317] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:55:03 2022] DELETE /identity/v3/projects/8bc7ab60b5d344b6a762bde55105e04a => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:55:03.886798 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-052f3e11-6960-4c80-96c2-bcc23af6ee56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f329efa85eee4255b208df73dafb3732', 'request_id': 'req-052f3e11-6960-4c80-96c2-bcc23af6ee56'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:03.887669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9493d4-ab9d-4a91-8de2-1c61881f4c0b None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:03.887993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9493d4-ab9d-4a91-8de2-1c61881f4c0b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:03.888418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9493d4-ab9d-4a91-8de2-1c61881f4c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:03.894065 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-052f3e11-6960-4c80-96c2-bcc23af6ee56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f329efa85eee4255b208df73dafb3732', 'request_id': 'req-052f3e11-6960-4c80-96c2-bcc23af6ee56'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:03.912188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-052f3e11-6960-4c80-96c2-bcc23af6ee56 admin admin] The token cache is being invalidate because project f329efa85eee4255b208df73dafb3732 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:55:03.912795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-052f3e11-6960-4c80-96c2-bcc23af6ee56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:03.914034 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-052f3e11-6960-4c80-96c2-bcc23af6ee56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:03.914533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-4c9493d4-ab9d-4a91-8de2-1c61881f4c0b None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:03.916007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4c9493d4-ab9d-4a91-8de2-1c61881f4c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:03.917600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1156/2318] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:55:03 2022] DELETE /identity/v3/projects/f329efa85eee4255b208df73dafb3732 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:03.923187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e6bae-f211-459f-ba2d-29448cea7832 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:03.923684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e6bae-f211-459f-ba2d-29448cea7832 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:03.924218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e6bae-f211-459f-ba2d-29448cea7832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:03.962676 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-d14e6bae-f211-459f-ba2d-29448cea7832 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:03.963981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d14e6bae-f211-459f-ba2d-29448cea7832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:04.044130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1163/2319] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:55:04.046876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bf7bf-b60f-481b-9c25-3d0af0a55408 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.047267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bf7bf-b60f-481b-9c25-3d0af0a55408 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.047614 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584bf7bf-b60f-481b-9c25-3d0af0a55408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.065192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1157/2320] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:04.074622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-584bf7bf-b60f-481b-9c25-3d0af0a55408 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:04.077268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-584bf7bf-b60f-481b-9c25-3d0af0a55408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:04.093479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1164/2321] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:04.100433 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e6bae-f211-459f-ba2d-29448cea7832 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:04.119589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e6bae-f211-459f-ba2d-29448cea7832 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:04.126551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5152b1-3065-4b62-8929-31bfb20b751b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:04.128054 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5152b1-3065-4b62-8929-31bfb20b751b admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.128413 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5152b1-3065-4b62-8929-31bfb20b751b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.128760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5152b1-3065-4b62-8929-31bfb20b751b admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.129651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5152b1-3065-4b62-8929-31bfb20b751b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:55:04.130601 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5152b1-3065-4b62-8929-31bfb20b751b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:04.132667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5152b1-3065-4b62-8929-31bfb20b751b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:04.135519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22281e47-68eb-4ba9-bb9b-474cc65fea96 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.136034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22281e47-68eb-4ba9-bb9b-474cc65fea96 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.136531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22281e47-68eb-4ba9-bb9b-474cc65fea96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.153078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1158/2322] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:55:04.158326 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5152b1-3065-4b62-8929-31bfb20b751b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:04.166747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-22281e47-68eb-4ba9-bb9b-474cc65fea96 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:04.168614 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-22281e47-68eb-4ba9-bb9b-474cc65fea96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:04.173796 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5152b1-3065-4b62-8929-31bfb20b751b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:04.181033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77966ccf-f587-429e-b427-2e0904937b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:04.181726 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1165/2323] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:55:04.182075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77966ccf-f587-429e-b427-2e0904937b35 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.182563 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77966ccf-f587-429e-b427-2e0904937b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.183045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77966ccf-f587-429e-b427-2e0904937b35 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.183826 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77966ccf-f587-429e-b427-2e0904937b35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:04.187019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77966ccf-f587-429e-b427-2e0904937b35 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:04.187216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22281e47-68eb-4ba9-bb9b-474cc65fea96 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:04.200198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22281e47-68eb-4ba9-bb9b-474cc65fea96 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:04.209060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fcc85-9317-491b-a854-b2248704578c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:04.210012 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fcc85-9317-491b-a854-b2248704578c admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.210492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fcc85-9317-491b-a854-b2248704578c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.210901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fcc85-9317-491b-a854-b2248704578c admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.211845 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fcc85-9317-491b-a854-b2248704578c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:55:04.212353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fcc85-9317-491b-a854-b2248704578c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:04.214445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fcc85-9317-491b-a854-b2248704578c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:04.230475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:04.238228 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1166/2324] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:55:04.245056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fcc85-9317-491b-a854-b2248704578c admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:04.248146 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1159/2325] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:55:04 2022] POST /identity/v3/projects => generated 409 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:55:04.253565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77966ccf-f587-429e-b427-2e0904937b35 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:04.264834 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fcc85-9317-491b-a854-b2248704578c admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:04.343096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:04.361617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4720df-665d-47f3-8346-c260acce5f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:04.362459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4720df-665d-47f3-8346-c260acce5f0b admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.362891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4720df-665d-47f3-8346-c260acce5f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.363299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4720df-665d-47f3-8346-c260acce5f0b admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.364971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4720df-665d-47f3-8346-c260acce5f0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:04.367057 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4720df-665d-47f3-8346-c260acce5f0b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:04.385946 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77966ccf-f587-429e-b427-2e0904937b35 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:04.395193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132598aa-bc95-4637-af77-48ab92c5315a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:04.396527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132598aa-bc95-4637-af77-48ab92c5315a admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.397012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132598aa-bc95-4637-af77-48ab92c5315a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.397674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132598aa-bc95-4637-af77-48ab92c5315a admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.398949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132598aa-bc95-4637-af77-48ab92c5315a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:04.402820 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132598aa-bc95-4637-af77-48ab92c5315a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:04.443204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1167/2326] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:55:04 2022] POST /identity/v3/projects => generated 391 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:55:04.448902 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4720df-665d-47f3-8346-c260acce5f0b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:04.456887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1160/2327] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:55:04 2022] POST /identity/v3/users => generated 305 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:55:04.459901 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132598aa-bc95-4637-af77-48ab92c5315a admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:04.484769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132598aa-bc95-4637-af77-48ab92c5315a admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:04.586333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4720df-665d-47f3-8346-c260acce5f0b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:04.594587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8bffb-2398-4b2f-bbd2-6808dbff4c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:04.595706 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8bffb-2398-4b2f-bbd2-6808dbff4c86 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.596235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8bffb-2398-4b2f-bbd2-6808dbff4c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.596895 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8bffb-2398-4b2f-bbd2-6808dbff4c86 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.597696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8bffb-2398-4b2f-bbd2-6808dbff4c86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:04.601010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8bffb-2398-4b2f-bbd2-6808dbff4c86 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:04.610322 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f75b3-b8eb-4072-a116-5054ef5e0d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:04.611491 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f75b3-b8eb-4072-a116-5054ef5e0d41 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.612052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f75b3-b8eb-4072-a116-5054ef5e0d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.612566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f75b3-b8eb-4072-a116-5054ef5e0d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.613836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f75b3-b8eb-4072-a116-5054ef5e0d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:04.617601 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f75b3-b8eb-4072-a116-5054ef5e0d41 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:04.632298 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1168/2328] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:55:04 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:55:04.635659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1161/2329] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:55:04.640941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f75b3-b8eb-4072-a116-5054ef5e0d41 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:04.641210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8bffb-2398-4b2f-bbd2-6808dbff4c86 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:04.653899 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8bffb-2398-4b2f-bbd2-6808dbff4c86 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:04.656746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f75b3-b8eb-4072-a116-5054ef5e0d41 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:04.660484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d023b-dcea-4353-a41e-5cfeb69ad731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:04.661972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d023b-dcea-4353-a41e-5cfeb69ad731 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.662326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d023b-dcea-4353-a41e-5cfeb69ad731 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.662670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d023b-dcea-4353-a41e-5cfeb69ad731 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.663513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d023b-dcea-4353-a41e-5cfeb69ad731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:04.664886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00d5c3-be15-432b-8e95-80ba9678cc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:04.665785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00d5c3-be15-432b-8e95-80ba9678cc3e admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.665969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01d023b-dcea-4353-a41e-5cfeb69ad731 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:04.666289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00d5c3-be15-432b-8e95-80ba9678cc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.666883 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc00d5c3-be15-432b-8e95-80ba9678cc3e admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.667984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00d5c3-be15-432b-8e95-80ba9678cc3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:04.670114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc00d5c3-be15-432b-8e95-80ba9678cc3e admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:04.681107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1169/2330] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:55:04.687207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d023b-dcea-4353-a41e-5cfeb69ad731 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:04.693753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1162/2331] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:55:04.698988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00d5c3-be15-432b-8e95-80ba9678cc3e admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:04.699675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d023b-dcea-4353-a41e-5cfeb69ad731 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:04.707419 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16642ba3-df82-420f-89c1-bf826155af99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:04.708675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16642ba3-df82-420f-89c1-bf826155af99 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.709052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16642ba3-df82-420f-89c1-bf826155af99 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.709410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16642ba3-df82-420f-89c1-bf826155af99 admin admin] PATH_INFO: `/v3/projects/ef58cfcb4faf4e2db269ce3d3b5122d7/users/9b9e3f541db24fae95a94eb9844496ea/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.719786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc00d5c3-be15-432b-8e95-80ba9678cc3e admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:04.731063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d3f7f-a4d1-462c-8f24-41b5f153ded0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:04.732255 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3f7f-a4d1-462c-8f24-41b5f153ded0 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.732917 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3f7f-a4d1-462c-8f24-41b5f153ded0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.733565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d3f7f-a4d1-462c-8f24-41b5f153ded0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.734536 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16642ba3-df82-420f-89c1-bf826155af99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef58cfcb4faf4e2db269ce3d3b5122d7, user_id=9b9e3f541db24fae95a94eb9844496ea, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:04.735188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d3f7f-a4d1-462c-8f24-41b5f153ded0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:04.737447 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16642ba3-df82-420f-89c1-bf826155af99 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:04.739067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011d3f7f-a4d1-462c-8f24-41b5f153ded0 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:04.764312 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1170/2332] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:55:04 2022] PUT /identity/v3/projects/ef58cfcb4faf4e2db269ce3d3b5122d7/users/9b9e3f541db24fae95a94eb9844496ea/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:04.771623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1163/2333] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 17:55:04.773982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a6c6b-a67c-49de-930b-eaea1a2f8209 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.776224 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d3f7f-a4d1-462c-8f24-41b5f153ded0 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:04.776484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a6c6b-a67c-49de-930b-eaea1a2f8209 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.777025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a6c6b-a67c-49de-930b-eaea1a2f8209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.797080 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d3f7f-a4d1-462c-8f24-41b5f153ded0 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:04.811561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f42a6c6b-a67c-49de-930b-eaea1a2f8209 None None] MFA Rules not processed for user `9b9e3f541db24fae95a94eb9844496ea`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:04.813157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f42a6c6b-a67c-49de-930b-eaea1a2f8209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:04.867560 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:04.923988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1171/2334] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:04 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:04.935047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a6c6b-a67c-49de-930b-eaea1a2f8209 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:04.935918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e016a-0c43-4ca1-ab3e-465783205252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:04.937439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e016a-0c43-4ca1-ab3e-465783205252 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.938053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e016a-0c43-4ca1-ab3e-465783205252 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.938528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e016a-0c43-4ca1-ab3e-465783205252 admin admin] PATH_INFO: `/v3/projects/8fb74e743b144c62b820b61c79a6de10/users/0aa0e9884a554430afae011ca765da05/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.953597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a6c6b-a67c-49de-930b-eaea1a2f8209 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:04.971089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e80613-e034-41d4-a0ca-9c394316dd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:04.974186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e016a-0c43-4ca1-ab3e-465783205252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fb74e743b144c62b820b61c79a6de10, user_id=0aa0e9884a554430afae011ca765da05, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:04.975897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e80613-e034-41d4-a0ca-9c394316dd23 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:04.979440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e016a-0c43-4ca1-ab3e-465783205252 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:04.982562 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e80613-e034-41d4-a0ca-9c394316dd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:04.983451 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e80613-e034-41d4-a0ca-9c394316dd23 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:04.985683 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e80613-e034-41d4-a0ca-9c394316dd23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:04.991019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e80613-e034-41d4-a0ca-9c394316dd23 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:05.009962 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1164/2335] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:55:04 2022] PUT /identity/v3/projects/8fb74e743b144c62b820b61c79a6de10/users/0aa0e9884a554430afae011ca765da05/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:05.029417 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bc39f-9165-451d-ac10-28a24e312d2a None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:05.030029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bc39f-9165-451d-ac10-28a24e312d2a None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:05.030507 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bc39f-9165-451d-ac10-28a24e312d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:05.083391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-248bc39f-9165-451d-ac10-28a24e312d2a None None] MFA Rules not processed for user `0aa0e9884a554430afae011ca765da05`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:05.084420 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1172/2336] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:55:04 2022] POST /identity/v3/projects => generated 409 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:55:05.085729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-248bc39f-9165-451d-ac10-28a24e312d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:05.089102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e80613-e034-41d4-a0ca-9c394316dd23 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:05.137053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:05.213456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1165/2337] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:05.223662 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e80613-e034-41d4-a0ca-9c394316dd23 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:05.226948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bc39f-9165-451d-ac10-28a24e312d2a None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:05.228703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-248bc39f-9165-451d-ac10-28a24e312d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:05.229673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416fcfa-c286-41b7-9ff1-743de192f5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:05.230537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416fcfa-c286-41b7-9ff1-743de192f5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:05.230888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416fcfa-c286-41b7-9ff1-743de192f5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:05.231242 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416fcfa-c286-41b7-9ff1-743de192f5a4 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:05.232149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416fcfa-c286-41b7-9ff1-743de192f5a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:05.234721 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416fcfa-c286-41b7-9ff1-743de192f5a4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:05.261983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1173/2338] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:55:05 2022] POST /identity/v3/users => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:55:05.267181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416fcfa-c286-41b7-9ff1-743de192f5a4 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:05.285137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416fcfa-c286-41b7-9ff1-743de192f5a4 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:05.295996 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98faed-2214-4103-8992-5d2f914d1872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:05.297190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98faed-2214-4103-8992-5d2f914d1872 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:05.297787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98faed-2214-4103-8992-5d2f914d1872 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:05.298222 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e98faed-2214-4103-8992-5d2f914d1872 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:05.299736 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98faed-2214-4103-8992-5d2f914d1872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:05.303096 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e98faed-2214-4103-8992-5d2f914d1872 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:05.313101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bc39f-9165-451d-ac10-28a24e312d2a None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:05.319343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97ce1e-0fce-417e-be10-fb893f501f88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:05.320415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97ce1e-0fce-417e-be10-fb893f501f88 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:05.320875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97ce1e-0fce-417e-be10-fb893f501f88 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:05.321399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97ce1e-0fce-417e-be10-fb893f501f88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:05.331439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1174/2339] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:55:05.334440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98faed-2214-4103-8992-5d2f914d1872 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:05.338478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97ce1e-0fce-417e-be10-fb893f501f88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:05.341297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af97ce1e-0fce-417e-be10-fb893f501f88 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:05.350912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e98faed-2214-4103-8992-5d2f914d1872 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:05.358532 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2505845-a7e3-405a-a5fc-d077d5f6ba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:05.360178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2505845-a7e3-405a-a5fc-d077d5f6ba91 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:05.360674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2505845-a7e3-405a-a5fc-d077d5f6ba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:05.361446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2505845-a7e3-405a-a5fc-d077d5f6ba91 admin admin] PATH_INFO: `/v3/projects/6f99ea3327694039a6a15f17482e1089/users/0cebefbd1161459ea9149e969bbd6d02/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:05.361624 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1166/2340] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:55:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:55:05.388729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2505845-a7e3-405a-a5fc-d077d5f6ba91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f99ea3327694039a6a15f17482e1089, user_id=0cebefbd1161459ea9149e969bbd6d02, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:05.394606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2505845-a7e3-405a-a5fc-d077d5f6ba91 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:05.421716 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1175/2341] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:55:05 2022] PUT /identity/v3/projects/6f99ea3327694039a6a15f17482e1089/users/0cebefbd1161459ea9149e969bbd6d02/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:55:05.428023 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97ce1e-0fce-417e-be10-fb893f501f88 service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:05.447208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af97ce1e-0fce-417e-be10-fb893f501f88 service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:05.559764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56f123f-3851-42f9-bec6-d24215fd7880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:05.560485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f123f-3851-42f9-bec6-d24215fd7880 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:05.560906 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f123f-3851-42f9-bec6-d24215fd7880 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:05.561867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56f123f-3851-42f9-bec6-d24215fd7880 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:05.562637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f123f-3851-42f9-bec6-d24215fd7880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:05.564776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56f123f-3851-42f9-bec6-d24215fd7880 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:05.583988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1167/2342] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:55:05.590383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2505845-a7e3-405a-a5fc-d077d5f6ba91 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:05.601827 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2505845-a7e3-405a-a5fc-d077d5f6ba91 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:05.608675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfff361-b2f1-4801-aa27-0c23cac58882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:05.610537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfff361-b2f1-4801-aa27-0c23cac58882 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:05.611222 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfff361-b2f1-4801-aa27-0c23cac58882 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:05.612037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfff361-b2f1-4801-aa27-0c23cac58882 admin admin] PATH_INFO: `/v3/projects/6f99ea3327694039a6a15f17482e1089/users/0cebefbd1161459ea9149e969bbd6d02/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:05.616854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfff361-b2f1-4801-aa27-0c23cac58882 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f99ea3327694039a6a15f17482e1089, user_id=0cebefbd1161459ea9149e969bbd6d02, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:05.620495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfff361-b2f1-4801-aa27-0c23cac58882 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:05.636968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1176/2343] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:55:05 2022] PUT /identity/v3/projects/6f99ea3327694039a6a15f17482e1089/users/0cebefbd1161459ea9149e969bbd6d02/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:05.650757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e6d48-da06-4635-9218-4bb4cfcb2022 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:05.651426 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e6d48-da06-4635-9218-4bb4cfcb2022 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:05.653012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319e6d48-da06-4635-9218-4bb4cfcb2022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:05.713416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-319e6d48-da06-4635-9218-4bb4cfcb2022 None None] MFA Rules not processed for user `0cebefbd1161459ea9149e969bbd6d02`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:05.717280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-319e6d48-da06-4635-9218-4bb4cfcb2022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:05.814854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:05.866075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1168/2344] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:05 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:05.880995 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfff361-b2f1-4801-aa27-0c23cac58882 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:05.939139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:05.943501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfff361-b2f1-4801-aa27-0c23cac58882 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:05.948265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728d3db-7018-4c79-81ec-ed494f8c1cf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:05.949094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728d3db-7018-4c79-81ec-ed494f8c1cf9 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:05.949380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728d3db-7018-4c79-81ec-ed494f8c1cf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:05.949661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728d3db-7018-4c79-81ec-ed494f8c1cf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:05.961182 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728d3db-7018-4c79-81ec-ed494f8c1cf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:05.963213 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728d3db-7018-4c79-81ec-ed494f8c1cf9 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:05.975442 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1177/2345] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:55:05 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:55:06.205782 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e6d48-da06-4635-9218-4bb4cfcb2022 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:06.208551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-319e6d48-da06-4635-9218-4bb4cfcb2022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:06.225726 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319e6d48-da06-4635-9218-4bb4cfcb2022 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:06.229993 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a273a1-f7da-4c38-a2df-be16f1eadd87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:06.230614 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a273a1-f7da-4c38-a2df-be16f1eadd87 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:06.231009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a273a1-f7da-4c38-a2df-be16f1eadd87 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:06.231406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a273a1-f7da-4c38-a2df-be16f1eadd87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:06.332956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a273a1-f7da-4c38-a2df-be16f1eadd87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:06.334939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a273a1-f7da-4c38-a2df-be16f1eadd87 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:06.358722 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1169/2346] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:55:06 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:55:07.719698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728d3db-7018-4c79-81ec-ed494f8c1cf9 service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:07.721207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c728d3db-7018-4c79-81ec-ed494f8c1cf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:07.823488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728d3db-7018-4c79-81ec-ed494f8c1cf9 service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:07.829074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d382267-675a-40a3-aa24-6a3e6023769e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:07.829888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d382267-675a-40a3-aa24-6a3e6023769e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:07.830267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d382267-675a-40a3-aa24-6a3e6023769e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:07.830445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d382267-675a-40a3-aa24-6a3e6023769e admin admin] PATH_INFO: `/v3/users/bb3d8225603e411e847581fec95a16bb` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:07.831880 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d382267-675a-40a3-aa24-6a3e6023769e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb3d8225603e411e847581fec95a16bb)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:07.834854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d382267-675a-40a3-aa24-6a3e6023769e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:07.902754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1d382267-675a-40a3-aa24-6a3e6023769e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb3d8225603e411e847581fec95a16bb'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:07.963842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1d382267-675a-40a3-aa24-6a3e6023769e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb3d8225603e411e847581fec95a16bb'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:07.985016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1d382267-675a-40a3-aa24-6a3e6023769e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb3d8225603e411e847581fec95a16bb'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:07.987113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1d382267-675a-40a3-aa24-6a3e6023769e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb3d8225603e411e847581fec95a16bb'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:08.004560 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1d382267-675a-40a3-aa24-6a3e6023769e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb3d8225603e411e847581fec95a16bb'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:08.023318 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1178/2347] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:55:07 2022] DELETE /identity/v3/users/bb3d8225603e411e847581fec95a16bb => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:55:08.031535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130623fe-599b-4538-84d1-2b2275dd2698 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:08.032318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130623fe-599b-4538-84d1-2b2275dd2698 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:08.032822 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130623fe-599b-4538-84d1-2b2275dd2698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:08.067456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-130623fe-599b-4538-84d1-2b2275dd2698 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:08.070327 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-130623fe-599b-4538-84d1-2b2275dd2698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:08.267572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1170/2348] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:08.285019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d382267-675a-40a3-aa24-6a3e6023769e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:08.288281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1d382267-675a-40a3-aa24-6a3e6023769e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:08.434585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d382267-675a-40a3-aa24-6a3e6023769e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:08.439014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5094653-5102-4933-a880-43c9e73c342b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:08.439945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5094653-5102-4933-a880-43c9e73c342b service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:08.440333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5094653-5102-4933-a880-43c9e73c342b service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:08.440720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5094653-5102-4933-a880-43c9e73c342b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:08.455135 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5094653-5102-4933-a880-43c9e73c342b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:08.459162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5094653-5102-4933-a880-43c9e73c342b service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:08.479074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1179/2349] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:55:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:55:08.972377 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130623fe-599b-4538-84d1-2b2275dd2698 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:08.974126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-130623fe-599b-4538-84d1-2b2275dd2698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:08.987636 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130623fe-599b-4538-84d1-2b2275dd2698 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:08.993660 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998fc08-d35b-4e97-9437-dff7ae07e4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:08.994751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998fc08-d35b-4e97-9437-dff7ae07e4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:08.995035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998fc08-d35b-4e97-9437-dff7ae07e4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:08.995347 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998fc08-d35b-4e97-9437-dff7ae07e4fc admin admin] PATH_INFO: `/v3/projects/ba0533a36cbc4887b71ff0b351ce5996` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:08.996786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998fc08-d35b-4e97-9437-dff7ae07e4fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba0533a36cbc4887b71ff0b351ce5996)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:08.998788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998fc08-d35b-4e97-9437-dff7ae07e4fc admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:09.059192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0998fc08-d35b-4e97-9437-dff7ae07e4fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba0533a36cbc4887b71ff0b351ce5996', 'request_id': 'req-0998fc08-d35b-4e97-9437-dff7ae07e4fc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:09.069555 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0998fc08-d35b-4e97-9437-dff7ae07e4fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba0533a36cbc4887b71ff0b351ce5996', 'request_id': 'req-0998fc08-d35b-4e97-9437-dff7ae07e4fc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:09.077539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0998fc08-d35b-4e97-9437-dff7ae07e4fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba0533a36cbc4887b71ff0b351ce5996', 'request_id': 'req-0998fc08-d35b-4e97-9437-dff7ae07e4fc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:09.091612 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0998fc08-d35b-4e97-9437-dff7ae07e4fc admin admin] The token cache is being invalidate because project ba0533a36cbc4887b71ff0b351ce5996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:55:09.091850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0998fc08-d35b-4e97-9437-dff7ae07e4fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:09.092858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0998fc08-d35b-4e97-9437-dff7ae07e4fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:09.095200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1171/2350] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:55:08 2022] DELETE /identity/v3/projects/ba0533a36cbc4887b71ff0b351ce5996 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:18.469653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=83,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:19.087619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:33.389072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5094653-5102-4933-a880-43c9e73c342b service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:33.391375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b5094653-5102-4933-a880-43c9e73c342b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:33.414367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5094653-5102-4933-a880-43c9e73c342b service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:33.505988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829407e3-98f7-4fea-a78f-8d76b501a354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:33.507147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829407e3-98f7-4fea-a78f-8d76b501a354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:33.507540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829407e3-98f7-4fea-a78f-8d76b501a354 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:33.507951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829407e3-98f7-4fea-a78f-8d76b501a354 admin admin] PATH_INFO: `/v3/users/9b9e3f541db24fae95a94eb9844496ea` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:33.509476 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829407e3-98f7-4fea-a78f-8d76b501a354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b9e3f541db24fae95a94eb9844496ea)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:33.511813 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829407e3-98f7-4fea-a78f-8d76b501a354 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:33.585969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-829407e3-98f7-4fea-a78f-8d76b501a354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b9e3f541db24fae95a94eb9844496ea'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:33.604266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-829407e3-98f7-4fea-a78f-8d76b501a354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b9e3f541db24fae95a94eb9844496ea'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:33.614718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-829407e3-98f7-4fea-a78f-8d76b501a354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b9e3f541db24fae95a94eb9844496ea'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:33.615997 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-829407e3-98f7-4fea-a78f-8d76b501a354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b9e3f541db24fae95a94eb9844496ea'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:33.632227 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-829407e3-98f7-4fea-a78f-8d76b501a354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b9e3f541db24fae95a94eb9844496ea'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:33.641350 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1180/2351] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:55:33 2022] DELETE /identity/v3/users/9b9e3f541db24fae95a94eb9844496ea => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:33.647602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11745a5-21e1-4c24-8bbe-436f48ad5082 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:33.648575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11745a5-21e1-4c24-8bbe-436f48ad5082 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:33.648892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11745a5-21e1-4c24-8bbe-436f48ad5082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:33.677863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-b11745a5-21e1-4c24-8bbe-436f48ad5082 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:33.680023 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b11745a5-21e1-4c24-8bbe-436f48ad5082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:33.779343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1172/2352] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:33.792138 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829407e3-98f7-4fea-a78f-8d76b501a354 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:33.794650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-829407e3-98f7-4fea-a78f-8d76b501a354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:33.884892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829407e3-98f7-4fea-a78f-8d76b501a354 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:33.890236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2febb-3a9a-4758-8b91-f41f86f3ef94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:33.891359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2febb-3a9a-4758-8b91-f41f86f3ef94 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:33.891849 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2febb-3a9a-4758-8b91-f41f86f3ef94 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:33.892383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2febb-3a9a-4758-8b91-f41f86f3ef94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:33.908114 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2febb-3a9a-4758-8b91-f41f86f3ef94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:33.909781 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2febb-3a9a-4758-8b91-f41f86f3ef94 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:33.922659 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1181/2353] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:55:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:55:34.370937 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11745a5-21e1-4c24-8bbe-436f48ad5082 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:34.373453 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b11745a5-21e1-4c24-8bbe-436f48ad5082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:34.404820 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11745a5-21e1-4c24-8bbe-436f48ad5082 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:34.412280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:34.414862 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:34.416181 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:34.416559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] PATH_INFO: `/v3/projects/ef58cfcb4faf4e2db269ce3d3b5122d7` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:34.419000 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef58cfcb4faf4e2db269ce3d3b5122d7)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:34.422790 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:34.509482 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef58cfcb4faf4e2db269ce3d3b5122d7', 'request_id': 'req-a766ea40-a1f2-48cb-b577-751d320b8bb6'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:34.523604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef58cfcb4faf4e2db269ce3d3b5122d7', 'request_id': 'req-a766ea40-a1f2-48cb-b577-751d320b8bb6'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:34.534751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef58cfcb4faf4e2db269ce3d3b5122d7', 'request_id': 'req-a766ea40-a1f2-48cb-b577-751d320b8bb6'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:34.560363 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] The token cache is being invalidate because project ef58cfcb4faf4e2db269ce3d3b5122d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:55:34.560869 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:34.562116 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:34.565424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1173/2354] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:55:34 2022] DELETE /identity/v3/projects/ef58cfcb4faf4e2db269ce3d3b5122d7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:34.572901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2febb-3a9a-4758-8b91-f41f86f3ef94 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:34.591781 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2febb-3a9a-4758-8b91-f41f86f3ef94 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:34.632563 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:34.681864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628c4ad5-7ef3-44a2-af0c-96a3e3c2801d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:34.682862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c4ad5-7ef3-44a2-af0c-96a3e3c2801d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:34.683201 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c4ad5-7ef3-44a2-af0c-96a3e3c2801d admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:34.683523 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628c4ad5-7ef3-44a2-af0c-96a3e3c2801d admin admin] PATH_INFO: `/v3/users/0cebefbd1161459ea9149e969bbd6d02` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:34.684959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c4ad5-7ef3-44a2-af0c-96a3e3c2801d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cebefbd1161459ea9149e969bbd6d02)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:34.687079 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628c4ad5-7ef3-44a2-af0c-96a3e3c2801d admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:34.759512 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-628c4ad5-7ef3-44a2-af0c-96a3e3c2801d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cebefbd1161459ea9149e969bbd6d02'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:34.779500 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-628c4ad5-7ef3-44a2-af0c-96a3e3c2801d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cebefbd1161459ea9149e969bbd6d02'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:34.790302 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-628c4ad5-7ef3-44a2-af0c-96a3e3c2801d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cebefbd1161459ea9149e969bbd6d02'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:34.791108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-628c4ad5-7ef3-44a2-af0c-96a3e3c2801d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cebefbd1161459ea9149e969bbd6d02'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:34.801604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-628c4ad5-7ef3-44a2-af0c-96a3e3c2801d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cebefbd1161459ea9149e969bbd6d02'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:34.810979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1182/2355] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:55:34 2022] DELETE /identity/v3/users/0cebefbd1161459ea9149e969bbd6d02 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:35.283311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:35.286164 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:35.332580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766ea40-a1f2-48cb-b577-751d320b8bb6 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:35.420751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f51315-8663-4404-bb47-4bbf7cc07891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:35.422252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f51315-8663-4404-bb47-4bbf7cc07891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:35.422755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f51315-8663-4404-bb47-4bbf7cc07891 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:35.423262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f51315-8663-4404-bb47-4bbf7cc07891 admin admin] PATH_INFO: `/v3/projects/6f99ea3327694039a6a15f17482e1089` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:35.425662 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f51315-8663-4404-bb47-4bbf7cc07891 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f99ea3327694039a6a15f17482e1089)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:35.430191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f51315-8663-4404-bb47-4bbf7cc07891 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:35.478519 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:35.506260 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-38f51315-8663-4404-bb47-4bbf7cc07891 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f99ea3327694039a6a15f17482e1089', 'request_id': 'req-38f51315-8663-4404-bb47-4bbf7cc07891'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:35.517893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-38f51315-8663-4404-bb47-4bbf7cc07891 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f99ea3327694039a6a15f17482e1089', 'request_id': 'req-38f51315-8663-4404-bb47-4bbf7cc07891'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:35.525994 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-38f51315-8663-4404-bb47-4bbf7cc07891 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f99ea3327694039a6a15f17482e1089', 'request_id': 'req-38f51315-8663-4404-bb47-4bbf7cc07891'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:35.548541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-38f51315-8663-4404-bb47-4bbf7cc07891 admin admin] The token cache is being invalidate because project 6f99ea3327694039a6a15f17482e1089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:55:35.549176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-38f51315-8663-4404-bb47-4bbf7cc07891 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:35.550400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-38f51315-8663-4404-bb47-4bbf7cc07891 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:35.552933 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1174/2356] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:55:35 2022] DELETE /identity/v3/projects/6f99ea3327694039a6a15f17482e1089 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:35.560746 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552b1a6-e642-41fc-afa2-2d51f2dce00a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:35.561308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552b1a6-e642-41fc-afa2-2d51f2dce00a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:35.561830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0552b1a6-e642-41fc-afa2-2d51f2dce00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:35.594775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-0552b1a6-e642-41fc-afa2-2d51f2dce00a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:35.597236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0552b1a6-e642-41fc-afa2-2d51f2dce00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:35.719396 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1183/2357] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:35.733776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336e058-56ec-4fe6-98dd-f2b46c0a59f0 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:35.734308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336e058-56ec-4fe6-98dd-f2b46c0a59f0 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:35.734794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336e058-56ec-4fe6-98dd-f2b46c0a59f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:35.768444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5336e058-56ec-4fe6-98dd-f2b46c0a59f0 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:35.770408 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5336e058-56ec-4fe6-98dd-f2b46c0a59f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:35.784733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1175/2358] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:35.791629 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552b1a6-e642-41fc-afa2-2d51f2dce00a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:35.802237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0552b1a6-e642-41fc-afa2-2d51f2dce00a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:35.808327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8d940-9014-49ee-8be4-2cb55ddeca44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:35.809119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8d940-9014-49ee-8be4-2cb55ddeca44 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:35.809508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8d940-9014-49ee-8be4-2cb55ddeca44 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:35.809856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b8d940-9014-49ee-8be4-2cb55ddeca44 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:35.810687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8d940-9014-49ee-8be4-2cb55ddeca44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:55:35.811069 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8d940-9014-49ee-8be4-2cb55ddeca44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:35.812856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b8d940-9014-49ee-8be4-2cb55ddeca44 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:35.830855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1184/2359] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 17:55:35.836571 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336e058-56ec-4fe6-98dd-f2b46c0a59f0 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:35.857735 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336e058-56ec-4fe6-98dd-f2b46c0a59f0 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:35.863029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766b29d-19ee-4373-b1cb-3c71515ca8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:35.863818 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766b29d-19ee-4373-b1cb-3c71515ca8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:35.864178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766b29d-19ee-4373-b1cb-3c71515ca8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:35.864463 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766b29d-19ee-4373-b1cb-3c71515ca8d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:35.865177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766b29d-19ee-4373-b1cb-3c71515ca8d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:35.867814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1766b29d-19ee-4373-b1cb-3c71515ca8d9 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:35.904124 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1176/2360] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:55:35 2022] POST /identity/v3/projects => generated 381 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:55:35.905485 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8d940-9014-49ee-8be4-2cb55ddeca44 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:35.917624 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b8d940-9014-49ee-8be4-2cb55ddeca44 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:35.957639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:35.997628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5f98c-93fd-4182-94f0-89a0f9068ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:35.998330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5f98c-93fd-4182-94f0-89a0f9068ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:35.998619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5f98c-93fd-4182-94f0-89a0f9068ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:35.998902 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5f98c-93fd-4182-94f0-89a0f9068ed4 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:35.999610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5f98c-93fd-4182-94f0-89a0f9068ed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:36.001636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5f98c-93fd-4182-94f0-89a0f9068ed4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:36.027125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1185/2361] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:55:35 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:55:36.031820 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766b29d-19ee-4373-b1cb-3c71515ca8d9 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:36.043776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766b29d-19ee-4373-b1cb-3c71515ca8d9 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:36.049043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21bd3c-8be7-49ba-8036-d7d980aad360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:36.049737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21bd3c-8be7-49ba-8036-d7d980aad360 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:36.050020 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21bd3c-8be7-49ba-8036-d7d980aad360 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:36.050303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a21bd3c-8be7-49ba-8036-d7d980aad360 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:36.051061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21bd3c-8be7-49ba-8036-d7d980aad360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:36.052775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a21bd3c-8be7-49ba-8036-d7d980aad360 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:36.066126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1177/2362] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:55:36.073955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5f98c-93fd-4182-94f0-89a0f9068ed4 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:36.088948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5f98c-93fd-4182-94f0-89a0f9068ed4 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:36.097428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fb804-13ee-43a9-8171-fd5668562797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:36.098368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fb804-13ee-43a9-8171-fd5668562797 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:36.098813 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fb804-13ee-43a9-8171-fd5668562797 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:36.099252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fb804-13ee-43a9-8171-fd5668562797 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:36.100721 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fb804-13ee-43a9-8171-fd5668562797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:36.103378 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fb804-13ee-43a9-8171-fd5668562797 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:36.116832 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1186/2363] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:55:36.123666 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21bd3c-8be7-49ba-8036-d7d980aad360 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:36.142009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a21bd3c-8be7-49ba-8036-d7d980aad360 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:36.148921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e260a-c6e8-45de-8825-328ac20668f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:36.150048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e260a-c6e8-45de-8825-328ac20668f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:36.150386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e260a-c6e8-45de-8825-328ac20668f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:36.150819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e260a-c6e8-45de-8825-328ac20668f2 admin admin] PATH_INFO: `/v3/projects/dd9f3d86a73f4e4f8ae0ee6082f2c765/users/9d5a58b180724eb4ae270e6b33a2949c/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:36.171130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e260a-c6e8-45de-8825-328ac20668f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd9f3d86a73f4e4f8ae0ee6082f2c765, user_id=9d5a58b180724eb4ae270e6b33a2949c, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:36.174081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e260a-c6e8-45de-8825-328ac20668f2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:36.190508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1178/2364] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:55:36 2022] PUT /identity/v3/projects/dd9f3d86a73f4e4f8ae0ee6082f2c765/users/9d5a58b180724eb4ae270e6b33a2949c/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:36.208652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52becd-4b91-4fed-a78c-af945db71eb5 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:36.209125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52becd-4b91-4fed-a78c-af945db71eb5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:36.209589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52becd-4b91-4fed-a78c-af945db71eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:36.242372 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-ae52becd-4b91-4fed-a78c-af945db71eb5 None None] MFA Rules not processed for user `9d5a58b180724eb4ae270e6b33a2949c`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:36.244290 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ae52becd-4b91-4fed-a78c-af945db71eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:36.356209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1187/2365] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:36.362089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e260a-c6e8-45de-8825-328ac20668f2 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:36.475064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e260a-c6e8-45de-8825-328ac20668f2 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:36.480514 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7d46c-36fe-44d3-9343-df036610e324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:36.481244 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7d46c-36fe-44d3-9343-df036610e324 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:36.481532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7d46c-36fe-44d3-9343-df036610e324 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:36.481814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7d46c-36fe-44d3-9343-df036610e324 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:36.482535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7d46c-36fe-44d3-9343-df036610e324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:36.484538 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7d46c-36fe-44d3-9343-df036610e324 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:36.515851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1179/2366] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:55:36 2022] POST /identity/v3/projects => generated 379 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:55:36.520310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52becd-4b91-4fed-a78c-af945db71eb5 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:36.532346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52becd-4b91-4fed-a78c-af945db71eb5 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:36.600186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad514c2-e1a0-4671-8941-0090afb330a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:36.601430 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad514c2-e1a0-4671-8941-0090afb330a4 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:36.601671 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad514c2-e1a0-4671-8941-0090afb330a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:36.601953 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad514c2-e1a0-4671-8941-0090afb330a4 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:36.602702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad514c2-e1a0-4671-8941-0090afb330a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:36.604752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad514c2-e1a0-4671-8941-0090afb330a4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:36.631181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1188/2367] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:55:36 2022] POST /identity/v3/users => generated 296 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:55:36.635574 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7d46c-36fe-44d3-9343-df036610e324 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:36.649605 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7d46c-36fe-44d3-9343-df036610e324 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:36.656825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d65586-ee34-44ba-a267-e98b334dbbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:36.657557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d65586-ee34-44ba-a267-e98b334dbbb9 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:36.657850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d65586-ee34-44ba-a267-e98b334dbbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:36.658127 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d65586-ee34-44ba-a267-e98b334dbbb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:36.658955 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d65586-ee34-44ba-a267-e98b334dbbb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:36.660636 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d65586-ee34-44ba-a267-e98b334dbbb9 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:36.673623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1180/2368] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:55:36.680694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad514c2-e1a0-4671-8941-0090afb330a4 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:36.692548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad514c2-e1a0-4671-8941-0090afb330a4 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:36.697850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba376824-0a90-4a52-afc1-1cda1a64273a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:36.698774 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba376824-0a90-4a52-afc1-1cda1a64273a admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:36.699060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba376824-0a90-4a52-afc1-1cda1a64273a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:36.699379 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba376824-0a90-4a52-afc1-1cda1a64273a admin admin] PATH_INFO: `/v3/projects/d8610ab227964f258f0bd9b979823b66/users/1bf8194a23d44a66842f72701c8fdc9a/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:36.719178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba376824-0a90-4a52-afc1-1cda1a64273a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8610ab227964f258f0bd9b979823b66, user_id=1bf8194a23d44a66842f72701c8fdc9a, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:36.722133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba376824-0a90-4a52-afc1-1cda1a64273a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:36.739023 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1189/2369] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:55:36 2022] PUT /identity/v3/projects/d8610ab227964f258f0bd9b979823b66/users/1bf8194a23d44a66842f72701c8fdc9a/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:55:36.744872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d65586-ee34-44ba-a267-e98b334dbbb9 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:36.756561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d65586-ee34-44ba-a267-e98b334dbbb9 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:36.782158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:36.828058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d105b-4eca-42d0-86c2-e9b6848d8358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:36.829018 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d105b-4eca-42d0-86c2-e9b6848d8358 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:36.829306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d105b-4eca-42d0-86c2-e9b6848d8358 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:36.829582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d105b-4eca-42d0-86c2-e9b6848d8358 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:36.830328 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d105b-4eca-42d0-86c2-e9b6848d8358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:36.832076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d105b-4eca-42d0-86c2-e9b6848d8358 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:36.844590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1181/2370] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:55:36.850071 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba376824-0a90-4a52-afc1-1cda1a64273a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:36.856561 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:36.865805 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba376824-0a90-4a52-afc1-1cda1a64273a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:36.876557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68e794-3512-495b-8de3-8807fe2e550c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:36.878143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68e794-3512-495b-8de3-8807fe2e550c admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:36.880695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68e794-3512-495b-8de3-8807fe2e550c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:36.881310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68e794-3512-495b-8de3-8807fe2e550c admin admin] PATH_INFO: `/v3/projects/d8610ab227964f258f0bd9b979823b66/users/1bf8194a23d44a66842f72701c8fdc9a/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:36.886388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68e794-3512-495b-8de3-8807fe2e550c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8610ab227964f258f0bd9b979823b66, user_id=1bf8194a23d44a66842f72701c8fdc9a, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:36.892271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68e794-3512-495b-8de3-8807fe2e550c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:36.913149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1190/2371] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:55:36 2022] PUT /identity/v3/projects/d8610ab227964f258f0bd9b979823b66/users/1bf8194a23d44a66842f72701c8fdc9a/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:36.934049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf988d-aec4-495d-bf26-333026790285 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:36.934553 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf988d-aec4-495d-bf26-333026790285 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:36.935060 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cf988d-aec4-495d-bf26-333026790285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:37.001575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-a9cf988d-aec4-495d-bf26-333026790285 None None] MFA Rules not processed for user `1bf8194a23d44a66842f72701c8fdc9a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:37.004266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a9cf988d-aec4-495d-bf26-333026790285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:37.194004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1182/2372] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:37.221468 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68e794-3512-495b-8de3-8807fe2e550c admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:37.224366 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8c68e794-3512-495b-8de3-8807fe2e550c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:37.269786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf988d-aec4-495d-bf26-333026790285 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:37.272042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a9cf988d-aec4-495d-bf26-333026790285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:37.312600 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68e794-3512-495b-8de3-8807fe2e550c admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:37.321365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cd3d8-454c-496f-8b27-09c608dd3dd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:37.322498 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cd3d8-454c-496f-8b27-09c608dd3dd6 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:37.323011 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cd3d8-454c-496f-8b27-09c608dd3dd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:37.323560 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cd3d8-454c-496f-8b27-09c608dd3dd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:37.343204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7cd3d8-454c-496f-8b27-09c608dd3dd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:37.346034 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7cd3d8-454c-496f-8b27-09c608dd3dd6 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:37.369603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1191/2373] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:55:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:55:37.389902 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cf988d-aec4-495d-bf26-333026790285 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:37.398785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12eea3-c080-4e85-a573-290988b3edd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:37.399678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12eea3-c080-4e85-a573-290988b3edd7 service placement] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:37.400175 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12eea3-c080-4e85-a573-290988b3edd7 service placement] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:37.400605 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12eea3-c080-4e85-a573-290988b3edd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:37.403332 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-eb12eea3-c080-4e85-a573-290988b3edd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:37.425441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12eea3-c080-4e85-a573-290988b3edd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:37.427123 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12eea3-c080-4e85-a573-290988b3edd7 service placement] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:37.439798 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1183/2374] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:55:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:55:37.894806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cd3d8-454c-496f-8b27-09c608dd3dd6 service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:37.979340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cd3d8-454c-496f-8b27-09c608dd3dd6 service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:37.984105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b5063-879c-438c-910f-65ee5870647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:37.985019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b5063-879c-438c-910f-65ee5870647c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:37.985427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b5063-879c-438c-910f-65ee5870647c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:37.985612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b5063-879c-438c-910f-65ee5870647c admin admin] PATH_INFO: `/v3/users/9d5a58b180724eb4ae270e6b33a2949c` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:37.986928 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b5063-879c-438c-910f-65ee5870647c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d5a58b180724eb4ae270e6b33a2949c)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:37.988983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b5063-879c-438c-910f-65ee5870647c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:38.062437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-8b4b5063-879c-438c-910f-65ee5870647c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d5a58b180724eb4ae270e6b33a2949c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:38.084117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-8b4b5063-879c-438c-910f-65ee5870647c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d5a58b180724eb4ae270e6b33a2949c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:38.096504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-8b4b5063-879c-438c-910f-65ee5870647c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d5a58b180724eb4ae270e6b33a2949c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:38.097566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-8b4b5063-879c-438c-910f-65ee5870647c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d5a58b180724eb4ae270e6b33a2949c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:38.114213 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-8b4b5063-879c-438c-910f-65ee5870647c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d5a58b180724eb4ae270e6b33a2949c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:38.125181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1192/2375] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:55:37 2022] DELETE /identity/v3/users/9d5a58b180724eb4ae270e6b33a2949c => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:38.130774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533de3eb-893f-4c13-8e86-3174d5624db7 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:38.131323 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533de3eb-893f-4c13-8e86-3174d5624db7 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:38.131872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533de3eb-893f-4c13-8e86-3174d5624db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:38.154730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-533de3eb-893f-4c13-8e86-3174d5624db7 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:38.156009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-533de3eb-893f-4c13-8e86-3174d5624db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:38.192929 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:38.236087 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1184/2376] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:38.251684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b5063-879c-438c-910f-65ee5870647c admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:38.253907 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8b4b5063-879c-438c-910f-65ee5870647c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:38.282829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:38.343873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b5063-879c-438c-910f-65ee5870647c admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:38.348732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60407a-aba6-478a-8f3d-5ffe4b84eef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:38.349435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60407a-aba6-478a-8f3d-5ffe4b84eef9 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:38.349834 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60407a-aba6-478a-8f3d-5ffe4b84eef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:38.350020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60407a-aba6-478a-8f3d-5ffe4b84eef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:38.366257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f60407a-aba6-478a-8f3d-5ffe4b84eef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:38.368604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f60407a-aba6-478a-8f3d-5ffe4b84eef9 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:38.381296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1193/2377] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:55:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:55:38.688382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533de3eb-893f-4c13-8e86-3174d5624db7 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:38.690914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-533de3eb-893f-4c13-8e86-3174d5624db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:38.718200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533de3eb-893f-4c13-8e86-3174d5624db7 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:38.727774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b375c-c53b-4ed9-b33e-58fb226f9a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:38.731483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b375c-c53b-4ed9-b33e-58fb226f9a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:38.732785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b375c-c53b-4ed9-b33e-58fb226f9a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:38.733590 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b375c-c53b-4ed9-b33e-58fb226f9a37 admin admin] PATH_INFO: `/v3/users/0aa0e9884a554430afae011ca765da05` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:38.737419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b375c-c53b-4ed9-b33e-58fb226f9a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aa0e9884a554430afae011ca765da05)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:38.741662 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b375c-c53b-4ed9-b33e-58fb226f9a37 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:38.886154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c57b375c-c53b-4ed9-b33e-58fb226f9a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aa0e9884a554430afae011ca765da05'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:38.912307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c57b375c-c53b-4ed9-b33e-58fb226f9a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aa0e9884a554430afae011ca765da05'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:38.914809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c57b375c-c53b-4ed9-b33e-58fb226f9a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aa0e9884a554430afae011ca765da05'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:38.926889 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c57b375c-c53b-4ed9-b33e-58fb226f9a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aa0e9884a554430afae011ca765da05'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:38.928394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c57b375c-c53b-4ed9-b33e-58fb226f9a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aa0e9884a554430afae011ca765da05'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:38.947032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60407a-aba6-478a-8f3d-5ffe4b84eef9 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:38.949598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3f60407a-aba6-478a-8f3d-5ffe4b84eef9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:38.956974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1185/2378] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:55:38 2022] DELETE /identity/v3/users/0aa0e9884a554430afae011ca765da05 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:38.962400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095fbc5-8190-4800-a3b1-cdc25b2b7197 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:38.962918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095fbc5-8190-4800-a3b1-cdc25b2b7197 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:38.963485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095fbc5-8190-4800-a3b1-cdc25b2b7197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:38.965552 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60407a-aba6-478a-8f3d-5ffe4b84eef9 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:39.012731 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-8095fbc5-8190-4800-a3b1-cdc25b2b7197 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:39.019281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-8095fbc5-8190-4800-a3b1-cdc25b2b7197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:39.089030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:39.091073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:39.091073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:39.091764 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] PATH_INFO: `/v3/projects/dd9f3d86a73f4e4f8ae0ee6082f2c765` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:39.094483 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd9f3d86a73f4e4f8ae0ee6082f2c765)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:39.098508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:39.140049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1186/2379] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:39.161580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095fbc5-8190-4800-a3b1-cdc25b2b7197 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:39.168368 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-8095fbc5-8190-4800-a3b1-cdc25b2b7197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:39.193846 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd9f3d86a73f4e4f8ae0ee6082f2c765', 'request_id': 'req-5b228510-1c37-4365-bf78-eade7794370d'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:39.208742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd9f3d86a73f4e4f8ae0ee6082f2c765', 'request_id': 'req-5b228510-1c37-4365-bf78-eade7794370d'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:39.225758 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd9f3d86a73f4e4f8ae0ee6082f2c765', 'request_id': 'req-5b228510-1c37-4365-bf78-eade7794370d'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:39.248098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] The token cache is being invalidate because project dd9f3d86a73f4e4f8ae0ee6082f2c765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:55:39.248453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:39.249210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:39.252858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1194/2380] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:55:38 2022] DELETE /identity/v3/projects/dd9f3d86a73f4e4f8ae0ee6082f2c765 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:39.256890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:39.259010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:39.329332 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:39.345471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095fbc5-8190-4800-a3b1-cdc25b2b7197 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:39.352979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:39.413984 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b228510-1c37-4365-bf78-eade7794370d admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:39.423744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e11d4c-c0bd-4dbf-b81a-2ebb014a1ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:39.425508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e11d4c-c0bd-4dbf-b81a-2ebb014a1ec8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:39.425994 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e11d4c-c0bd-4dbf-b81a-2ebb014a1ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:39.427865 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e11d4c-c0bd-4dbf-b81a-2ebb014a1ec8 admin admin] PATH_INFO: `/v3/users/1bf8194a23d44a66842f72701c8fdc9a` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:39.430763 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e11d4c-c0bd-4dbf-b81a-2ebb014a1ec8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bf8194a23d44a66842f72701c8fdc9a)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:39.434350 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e11d4c-c0bd-4dbf-b81a-2ebb014a1ec8 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:39.480753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca43548-d187-4def-a568-57562cb87765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:39.481684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca43548-d187-4def-a568-57562cb87765 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:39.482108 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca43548-d187-4def-a568-57562cb87765 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:39.482663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca43548-d187-4def-a568-57562cb87765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:39.486278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-bca43548-d187-4def-a568-57562cb87765 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:39.505724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca43548-d187-4def-a568-57562cb87765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:39.507609 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca43548-d187-4def-a568-57562cb87765 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:39.524577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1187/2381] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:55:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:55:39.565771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-53e11d4c-c0bd-4dbf-b81a-2ebb014a1ec8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bf8194a23d44a66842f72701c8fdc9a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:39.596862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-53e11d4c-c0bd-4dbf-b81a-2ebb014a1ec8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bf8194a23d44a66842f72701c8fdc9a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:39.614792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-53e11d4c-c0bd-4dbf-b81a-2ebb014a1ec8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bf8194a23d44a66842f72701c8fdc9a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:39.615903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-53e11d4c-c0bd-4dbf-b81a-2ebb014a1ec8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bf8194a23d44a66842f72701c8fdc9a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:39.640670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-53e11d4c-c0bd-4dbf-b81a-2ebb014a1ec8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bf8194a23d44a66842f72701c8fdc9a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:39.664386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1195/2382] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:55:39 2022] DELETE /identity/v3/users/1bf8194a23d44a66842f72701c8fdc9a => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:40.160481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca43548-d187-4def-a568-57562cb87765 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:40.162779 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-bca43548-d187-4def-a568-57562cb87765 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:40.194122 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca43548-d187-4def-a568-57562cb87765 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:40.285422 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c3f4b-58f7-4739-87dd-dbc4c50684ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:40.286583 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c3f4b-58f7-4739-87dd-dbc4c50684ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:40.286865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c3f4b-58f7-4739-87dd-dbc4c50684ac admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:40.287143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c3f4b-58f7-4739-87dd-dbc4c50684ac admin admin] PATH_INFO: `/v3/projects/8fb74e743b144c62b820b61c79a6de10` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:40.289119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c3f4b-58f7-4739-87dd-dbc4c50684ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fb74e743b144c62b820b61c79a6de10)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:40.291167 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c3f4b-58f7-4739-87dd-dbc4c50684ac admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:40.370672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-698c3f4b-58f7-4739-87dd-dbc4c50684ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fb74e743b144c62b820b61c79a6de10', 'request_id': 'req-698c3f4b-58f7-4739-87dd-dbc4c50684ac'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:40.382906 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-698c3f4b-58f7-4739-87dd-dbc4c50684ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fb74e743b144c62b820b61c79a6de10', 'request_id': 'req-698c3f4b-58f7-4739-87dd-dbc4c50684ac'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:40.393165 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-698c3f4b-58f7-4739-87dd-dbc4c50684ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fb74e743b144c62b820b61c79a6de10', 'request_id': 'req-698c3f4b-58f7-4739-87dd-dbc4c50684ac'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:40.417721 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-698c3f4b-58f7-4739-87dd-dbc4c50684ac admin admin] The token cache is being invalidate because project 8fb74e743b144c62b820b61c79a6de10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:55:40.419072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-698c3f4b-58f7-4739-87dd-dbc4c50684ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:40.420515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-698c3f4b-58f7-4739-87dd-dbc4c50684ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:40.429100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1188/2383] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:55:40 2022] DELETE /identity/v3/projects/8fb74e743b144c62b820b61c79a6de10 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:40.550338 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e11d4c-c0bd-4dbf-b81a-2ebb014a1ec8 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:40.552870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-53e11d4c-c0bd-4dbf-b81a-2ebb014a1ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:40.576909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e11d4c-c0bd-4dbf-b81a-2ebb014a1ec8 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:40.660822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:40.661752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:40.662043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:40.662328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824 admin admin] PATH_INFO: `/v3/projects/d8610ab227964f258f0bd9b979823b66` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:40.664135 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8610ab227964f258f0bd9b979823b66)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:40.666211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:40.722081 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8610ab227964f258f0bd9b979823b66', 'request_id': 'req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:40.729869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8610ab227964f258f0bd9b979823b66', 'request_id': 'req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:40.737008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8610ab227964f258f0bd9b979823b66', 'request_id': 'req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:40.742220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:40.751189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824 admin admin] The token cache is being invalidate because project d8610ab227964f258f0bd9b979823b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:55:40.751499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:40.753045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1b8728ea-2ef7-4154-9cd3-58a1f2ae9824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:55:40.754795 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1196/2384] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:55:40 2022] DELETE /identity/v3/projects/d8610ab227964f258f0bd9b979823b66 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:55:40.762388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f525-6a2b-4293-acce-2fb843077cf3 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:40.762693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f525-6a2b-4293-acce-2fb843077cf3 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:40.762957 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac7f525-6a2b-4293-acce-2fb843077cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:40.783329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-7ac7f525-6a2b-4293-acce-2fb843077cf3 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:40.784603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7ac7f525-6a2b-4293-acce-2fb843077cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:40.822695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:40.855584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1189/2385] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:40.864417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b0169-389f-436e-94a9-20e7a7b532d7 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:40.864690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b0169-389f-436e-94a9-20e7a7b532d7 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:40.865022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b0169-389f-436e-94a9-20e7a7b532d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:40.885821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-559b0169-389f-436e-94a9-20e7a7b532d7 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:40.887078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-559b0169-389f-436e-94a9-20e7a7b532d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:40.895233 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1197/2386] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:40.899797 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7f525-6a2b-4293-acce-2fb843077cf3 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:40.910473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac7f525-6a2b-4293-acce-2fb843077cf3 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:40.916297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47468c6-f3bd-4f95-9d27-33f8ab5ad604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:40.917032 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47468c6-f3bd-4f95-9d27-33f8ab5ad604 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:40.917312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47468c6-f3bd-4f95-9d27-33f8ab5ad604 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:40.917592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47468c6-f3bd-4f95-9d27-33f8ab5ad604 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:40.918345 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47468c6-f3bd-4f95-9d27-33f8ab5ad604 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:55:40.918651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47468c6-f3bd-4f95-9d27-33f8ab5ad604 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:40.920318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47468c6-f3bd-4f95-9d27-33f8ab5ad604 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:40.934049 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1190/2387] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:55:40.938871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b0169-389f-436e-94a9-20e7a7b532d7 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:40.950583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b0169-389f-436e-94a9-20e7a7b532d7 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:40.956534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d038e7-edb6-4921-a4e8-ff9c3496fe10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:40.957274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d038e7-edb6-4921-a4e8-ff9c3496fe10 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:40.957680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d038e7-edb6-4921-a4e8-ff9c3496fe10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:40.958054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d038e7-edb6-4921-a4e8-ff9c3496fe10 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:40.958829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d038e7-edb6-4921-a4e8-ff9c3496fe10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:40.961533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d038e7-edb6-4921-a4e8-ff9c3496fe10 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:40.992469 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1198/2388] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:55:40 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:55:40.996881 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47468c6-f3bd-4f95-9d27-33f8ab5ad604 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:41.008666 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47468c6-f3bd-4f95-9d27-33f8ab5ad604 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:41.076457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba39b9-574c-41cc-bb26-b9a4d454ee17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:41.077164 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba39b9-574c-41cc-bb26-b9a4d454ee17 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:41.077456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba39b9-574c-41cc-bb26-b9a4d454ee17 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:41.077734 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba39b9-574c-41cc-bb26-b9a4d454ee17 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:41.078886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba39b9-574c-41cc-bb26-b9a4d454ee17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:41.081530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba39b9-574c-41cc-bb26-b9a4d454ee17 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:41.106221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1191/2389] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:55:40 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:55:41.111171 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d038e7-edb6-4921-a4e8-ff9c3496fe10 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:41.123534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d038e7-edb6-4921-a4e8-ff9c3496fe10 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:41.128918 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234d813-d56d-4ce7-bec6-cdfe265ce2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:41.129619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234d813-d56d-4ce7-bec6-cdfe265ce2be admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:41.129905 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234d813-d56d-4ce7-bec6-cdfe265ce2be admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:41.130187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b234d813-d56d-4ce7-bec6-cdfe265ce2be admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:41.131070 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234d813-d56d-4ce7-bec6-cdfe265ce2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:41.133280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b234d813-d56d-4ce7-bec6-cdfe265ce2be admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:41.146220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1199/2390] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:55:41.151561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba39b9-574c-41cc-bb26-b9a4d454ee17 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:41.162579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba39b9-574c-41cc-bb26-b9a4d454ee17 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:41.167763 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab02f6c-0e3b-44f5-8e76-f4f0068c007a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:41.168667 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab02f6c-0e3b-44f5-8e76-f4f0068c007a admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:41.168965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab02f6c-0e3b-44f5-8e76-f4f0068c007a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:41.169239 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab02f6c-0e3b-44f5-8e76-f4f0068c007a admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:41.169985 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab02f6c-0e3b-44f5-8e76-f4f0068c007a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:41.171623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab02f6c-0e3b-44f5-8e76-f4f0068c007a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:41.184578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1192/2391] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:55:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:55:41.189633 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234d813-d56d-4ce7-bec6-cdfe265ce2be admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:41.201015 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b234d813-d56d-4ce7-bec6-cdfe265ce2be admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:41.206518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6993642-c7de-4e2d-83e2-38d0343e0b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:41.207444 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6993642-c7de-4e2d-83e2-38d0343e0b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:41.207759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6993642-c7de-4e2d-83e2-38d0343e0b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:41.208055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6993642-c7de-4e2d-83e2-38d0343e0b6e admin admin] PATH_INFO: `/v3/projects/fa36825e7c014d48a7fe9375d9abdf7d/users/3e3fd98743554b0a8bf1cea0523ed4ae/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:41.226934 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6993642-c7de-4e2d-83e2-38d0343e0b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa36825e7c014d48a7fe9375d9abdf7d, user_id=3e3fd98743554b0a8bf1cea0523ed4ae, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:41.230010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6993642-c7de-4e2d-83e2-38d0343e0b6e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:41.245353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1200/2392] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:55:41 2022] PUT /identity/v3/projects/fa36825e7c014d48a7fe9375d9abdf7d/users/3e3fd98743554b0a8bf1cea0523ed4ae/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:41.251180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68285a35-b043-4c04-a30d-81ff48fde1fc None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:41.251515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68285a35-b043-4c04-a30d-81ff48fde1fc None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:41.251833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68285a35-b043-4c04-a30d-81ff48fde1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:41.272162 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-68285a35-b043-4c04-a30d-81ff48fde1fc None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:41.273943 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-68285a35-b043-4c04-a30d-81ff48fde1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:41.340898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1193/2393] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:41.351690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6993642-c7de-4e2d-83e2-38d0343e0b6e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:41.353489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d6993642-c7de-4e2d-83e2-38d0343e0b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:41.419805 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6993642-c7de-4e2d-83e2-38d0343e0b6e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:41.423644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a638d60-b0b0-4fc3-bfa5-80c87aaf1dc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:41.424355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a638d60-b0b0-4fc3-bfa5-80c87aaf1dc9 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:41.424638 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a638d60-b0b0-4fc3-bfa5-80c87aaf1dc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:41.424921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a638d60-b0b0-4fc3-bfa5-80c87aaf1dc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:41.435702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a638d60-b0b0-4fc3-bfa5-80c87aaf1dc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:41.437199 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a638d60-b0b0-4fc3-bfa5-80c87aaf1dc9 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:41.447895 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1201/2394] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:55:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:55:47.205038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f6237-3cf7-48d4-9446-a251b504e69f None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:47.205690 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f6237-3cf7-48d4-9446-a251b504e69f None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:47.206083 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f6237-3cf7-48d4-9446-a251b504e69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:47.256602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-142f6237-3cf7-48d4-9446-a251b504e69f None None] MFA Rules not processed for user `3e3fd98743554b0a8bf1cea0523ed4ae`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:47.259008 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-142f6237-3cf7-48d4-9446-a251b504e69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:47.306557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:47.368582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1194/2395] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:47.374172 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a638d60-b0b0-4fc3-bfa5-80c87aaf1dc9 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:47.377306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439af4db-398a-49bf-adde-234a923e99aa None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:47.377630 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439af4db-398a-49bf-adde-234a923e99aa None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:47.377971 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439af4db-398a-49bf-adde-234a923e99aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:47.391962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a638d60-b0b0-4fc3-bfa5-80c87aaf1dc9 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:47.399715 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94e761-5e4f-4fbc-b722-3afa9decb731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:47.400015 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-439af4db-398a-49bf-adde-234a923e99aa None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:47.401119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94e761-5e4f-4fbc-b722-3afa9decb731 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:47.401388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-439af4db-398a-49bf-adde-234a923e99aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:47.401772 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94e761-5e4f-4fbc-b722-3afa9decb731 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:47.402327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94e761-5e4f-4fbc-b722-3afa9decb731 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:47.403700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94e761-5e4f-4fbc-b722-3afa9decb731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:47.407219 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94e761-5e4f-4fbc-b722-3afa9decb731 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:47.464328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1202/2396] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:55:47 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 17:55:47.466846 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94e761-5e4f-4fbc-b722-3afa9decb731 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:47.558266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1195/2397] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 17:55:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:55:47.598450 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94e761-5e4f-4fbc-b722-3afa9decb731 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:47.606778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732e09f-4929-4919-997f-9053b546985d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:47.607587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732e09f-4929-4919-997f-9053b546985d admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:47.607966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732e09f-4929-4919-997f-9053b546985d admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:47.608313 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732e09f-4929-4919-997f-9053b546985d admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:47.609256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732e09f-4929-4919-997f-9053b546985d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:47.610299 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439af4db-398a-49bf-adde-234a923e99aa None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:47.611415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732e09f-4929-4919-997f-9053b546985d admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:47.612132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-439af4db-398a-49bf-adde-234a923e99aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:47.641282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:DELETE=1,keystone:SELECT=93,keystone:INSERT=6 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:47.649328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1203/2398] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:55:47 2022] POST /identity/v3/users => generated 299 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:55:47.650447 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732e09f-4929-4919-997f-9053b546985d admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:47.665227 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732e09f-4929-4919-997f-9053b546985d admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:47.673574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371de7f-64a1-4a23-8279-99718d7ba97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:47.674611 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371de7f-64a1-4a23-8279-99718d7ba97e admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:47.675007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371de7f-64a1-4a23-8279-99718d7ba97e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:47.675482 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1371de7f-64a1-4a23-8279-99718d7ba97e admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:47.676357 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371de7f-64a1-4a23-8279-99718d7ba97e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:47.678368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1371de7f-64a1-4a23-8279-99718d7ba97e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:47.693925 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1204/2399] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:55:47.698789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371de7f-64a1-4a23-8279-99718d7ba97e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:47.711552 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1371de7f-64a1-4a23-8279-99718d7ba97e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:47.718120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379bb55-5032-4d1b-8d6d-518664a06b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:47.719301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379bb55-5032-4d1b-8d6d-518664a06b38 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:47.719758 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379bb55-5032-4d1b-8d6d-518664a06b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:47.720118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d379bb55-5032-4d1b-8d6d-518664a06b38 admin admin] PATH_INFO: `/v3/projects/59e2d432a0c54303ae92759916a9fc37/users/79e6432ebc1c4eb496e5af7b8eabfab4/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:47.758843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439af4db-398a-49bf-adde-234a923e99aa None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:47.773257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379bb55-5032-4d1b-8d6d-518664a06b38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59e2d432a0c54303ae92759916a9fc37, user_id=79e6432ebc1c4eb496e5af7b8eabfab4, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:47.778985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d379bb55-5032-4d1b-8d6d-518664a06b38 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:47.809797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1205/2400] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:55:47 2022] PUT /identity/v3/projects/59e2d432a0c54303ae92759916a9fc37/users/79e6432ebc1c4eb496e5af7b8eabfab4/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:55:47.811174 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379bb55-5032-4d1b-8d6d-518664a06b38 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:47.880318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e0f2c1-b628-4e0b-9628-c259d25135ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:47.881233 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0f2c1-b628-4e0b-9628-c259d25135ef service swift] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:47.881416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0f2c1-b628-4e0b-9628-c259d25135ef service swift] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:47.881718 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e0f2c1-b628-4e0b-9628-c259d25135ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:47.901712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d379bb55-5032-4d1b-8d6d-518664a06b38 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:47.907279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebada5-15d5-457b-a354-6a0453840d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:47.908014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebada5-15d5-457b-a354-6a0453840d26 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:47.908327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebada5-15d5-457b-a354-6a0453840d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:47.908609 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebada5-15d5-457b-a354-6a0453840d26 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:47.909435 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebada5-15d5-457b-a354-6a0453840d26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:47.911047 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebada5-15d5-457b-a354-6a0453840d26 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:47.917720 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:55:47.923860 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1206/2401] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:55:47.930773 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebada5-15d5-457b-a354-6a0453840d26 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:55:47.943634 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebada5-15d5-457b-a354-6a0453840d26 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:55:47.949394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb60bf4-8dc1-4579-b89a-e8c9de07a2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:55:47.950486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb60bf4-8dc1-4579-b89a-e8c9de07a2fd admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:47.950775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb60bf4-8dc1-4579-b89a-e8c9de07a2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:47.951198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb60bf4-8dc1-4579-b89a-e8c9de07a2fd admin admin] PATH_INFO: `/v3/projects/59e2d432a0c54303ae92759916a9fc37/users/79e6432ebc1c4eb496e5af7b8eabfab4/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:47.953493 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb60bf4-8dc1-4579-b89a-e8c9de07a2fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=59e2d432a0c54303ae92759916a9fc37, user_id=79e6432ebc1c4eb496e5af7b8eabfab4, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:47.956813 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb60bf4-8dc1-4579-b89a-e8c9de07a2fd admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:47.970488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1207/2402] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:55:47 2022] PUT /identity/v3/projects/59e2d432a0c54303ae92759916a9fc37/users/79e6432ebc1c4eb496e5af7b8eabfab4/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:55:47.982789 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e0f2c1-b628-4e0b-9628-c259d25135ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:55:47.984700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e0f2c1-b628-4e0b-9628-c259d25135ef service swift] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:55:47.996149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1196/2403] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 17:55:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 17:55:55.902230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3908409-86b3-411f-9c35-58b66819b990 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:55:55.905496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3908409-86b3-411f-9c35-58b66819b990 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:55:55.906359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3908409-86b3-411f-9c35-58b66819b990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:55:55.953734 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-d3908409-86b3-411f-9c35-58b66819b990 None None] MFA Rules not processed for user `52c794c11ab74f9ca9bad75857043a62`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:55:55.957036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d3908409-86b3-411f-9c35-58b66819b990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:55:56.097420 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1208/2404] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:55:55 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:55:57.993592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=17 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:56:01.448112 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c3a43-3eef-4a76-a4c6-8a8133adc981 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:01.449371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c3a43-3eef-4a76-a4c6-8a8133adc981 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:01.449371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1c3a43-3eef-4a76-a4c6-8a8133adc981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:01.494775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-0c1c3a43-3eef-4a76-a4c6-8a8133adc981 None None] MFA Rules not processed for user `79e6432ebc1c4eb496e5af7b8eabfab4`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:56:01.497245 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0c1c3a43-3eef-4a76-a4c6-8a8133adc981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:01.690986 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1197/2405] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:56:01 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:56:01.717551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3908409-86b3-411f-9c35-58b66819b990 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:01.720213 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d3908409-86b3-411f-9c35-58b66819b990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:01.853334 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3908409-86b3-411f-9c35-58b66819b990 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:01.872393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5b957-00f3-4eca-bd70-9858c3af1dc6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2b4bf2aa1744a46b6d2a1b5a136a0e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:01.873684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b957-00f3-4eca-bd70-9858c3af1dc6 service glance] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:01.874246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b957-00f3-4eca-bd70-9858c3af1dc6 service glance] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:01.874741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5b957-00f3-4eca-bd70-9858c3af1dc6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:01.886549 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:56:02.048525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5b957-00f3-4eca-bd70-9858c3af1dc6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:02.065308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5b957-00f3-4eca-bd70-9858c3af1dc6 service glance] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:02.081332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1209/2406] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:56:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:56:02.437424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817b9b3-fe32-4da0-ad42-caf0a361ac68 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:02.437908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817b9b3-fe32-4da0-ad42-caf0a361ac68 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:02.438383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817b9b3-fe32-4da0-ad42-caf0a361ac68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:02.510772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-a817b9b3-fe32-4da0-ad42-caf0a361ac68 None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:56:02.512601 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a817b9b3-fe32-4da0-ad42-caf0a361ac68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:02.710500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1198/2407] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 17:56:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:56:02.731418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5b957-00f3-4eca-bd70-9858c3af1dc6 service glance] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:02.930181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5b957-00f3-4eca-bd70-9858c3af1dc6 service glance] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:02.944143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c296d09-1713-4de3-9fee-ffe4bb2b5ceb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:02.944939 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c296d09-1713-4de3-9fee-ffe4bb2b5ceb service swift] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:02.945285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c296d09-1713-4de3-9fee-ffe4bb2b5ceb service swift] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:02.945853 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c296d09-1713-4de3-9fee-ffe4bb2b5ceb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:02.957361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c296d09-1713-4de3-9fee-ffe4bb2b5ceb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:02.959159 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c296d09-1713-4de3-9fee-ffe4bb2b5ceb service swift] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:02.971733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1210/2408] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 17:56:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 17:56:12.671421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=59 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:56:12.966810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=56 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:56:19.154725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeed1b-9bd6-41a2-90ce-76c5b73489cc None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:19.155404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeed1b-9bd6-41a2-90ce-76c5b73489cc None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:19.156061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaeed1b-9bd6-41a2-90ce-76c5b73489cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:19.195730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-7eaeed1b-9bd6-41a2-90ce-76c5b73489cc None None] MFA Rules not processed for user `3e3fd98743554b0a8bf1cea0523ed4ae`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:56:19.197854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7eaeed1b-9bd6-41a2-90ce-76c5b73489cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:19.208859 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1199/2409] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:56:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:56:21.328141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874bde3-d022-4224-931a-4bb5f2da6c33 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:21.329489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874bde3-d022-4224-931a-4bb5f2da6c33 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:21.329968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1874bde3-d022-4224-931a-4bb5f2da6c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:21.358405 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-1874bde3-d022-4224-931a-4bb5f2da6c33 None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:56:21.361083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1874bde3-d022-4224-931a-4bb5f2da6c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:21.374400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1211/2410] 10.222.0.64 () {58 vars in 967 bytes} [Thu Jul 14 17:56:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:56:21.395158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaeed1b-9bd6-41a2-90ce-76c5b73489cc None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:21.426427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaeed1b-9bd6-41a2-90ce-76c5b73489cc None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:21.446048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c87969-c6bb-46f1-bbbf-30c6d80629bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:21.446740 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c87969-c6bb-46f1-bbbf-30c6d80629bc service swift] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:21.447027 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c87969-c6bb-46f1-bbbf-30c6d80629bc service swift] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:21.447395 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c87969-c6bb-46f1-bbbf-30c6d80629bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:21.458068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c87969-c6bb-46f1-bbbf-30c6d80629bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:21.459735 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c87969-c6bb-46f1-bbbf-30c6d80629bc service swift] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:21.469914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1200/2411] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 17:56:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 17:56:21.617728 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874bde3-d022-4224-931a-4bb5f2da6c33 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:21.716091 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1874bde3-d022-4224-931a-4bb5f2da6c33 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:21.720286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c6d11-23cd-448f-baf4-b5cdb030ca70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:21.721087 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c6d11-23cd-448f-baf4-b5cdb030ca70 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:21.721372 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c6d11-23cd-448f-baf4-b5cdb030ca70 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:21.721653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c6d11-23cd-448f-baf4-b5cdb030ca70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:21.723852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6d0c6d11-23cd-448f-baf4-b5cdb030ca70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:21.829648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c6d11-23cd-448f-baf4-b5cdb030ca70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:21.832898 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c6d11-23cd-448f-baf4-b5cdb030ca70 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:21.846818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1212/2412] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:56:21 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:56:22.644223 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45a6c4-0d74-4bb6-808e-15061b2253f3 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:22.647625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45a6c4-0d74-4bb6-808e-15061b2253f3 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:22.648425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45a6c4-0d74-4bb6-808e-15061b2253f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:22.671528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-dd45a6c4-0d74-4bb6-808e-15061b2253f3 None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:56:22.672991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-dd45a6c4-0d74-4bb6-808e-15061b2253f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:22.682904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1201/2413] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 17:56:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:56:22.713997 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c6d11-23cd-448f-baf4-b5cdb030ca70 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:22.737791 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c6d11-23cd-448f-baf4-b5cdb030ca70 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:22.749775 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c982790-403d-4e18-a9c8-67476fb83ff2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:22.751269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c982790-403d-4e18-a9c8-67476fb83ff2 service swift] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:22.752291 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c982790-403d-4e18-a9c8-67476fb83ff2 service swift] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:22.752933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c982790-403d-4e18-a9c8-67476fb83ff2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:22.770295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c982790-403d-4e18-a9c8-67476fb83ff2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:22.773741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c982790-403d-4e18-a9c8-67476fb83ff2 service swift] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:22.792409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1213/2414] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 17:56:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 17:56:24.490739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45a6c4-0d74-4bb6-808e-15061b2253f3 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:24.507793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-dd45a6c4-0d74-4bb6-808e-15061b2253f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:24.693318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45a6c4-0d74-4bb6-808e-15061b2253f3 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:24.704214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3062354-10a6-4712-809c-5aa798b7cfb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:24.705961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3062354-10a6-4712-809c-5aa798b7cfb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:24.706589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3062354-10a6-4712-809c-5aa798b7cfb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:24.707444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3062354-10a6-4712-809c-5aa798b7cfb3 admin admin] PATH_INFO: `/v3/users/d3b3a526943646dcba7bb1806601696b` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:24.713981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3062354-10a6-4712-809c-5aa798b7cfb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3b3a526943646dcba7bb1806601696b)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:24.717459 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3062354-10a6-4712-809c-5aa798b7cfb3 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:24.835916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d3062354-10a6-4712-809c-5aa798b7cfb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3b3a526943646dcba7bb1806601696b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:56:24.874952 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d3062354-10a6-4712-809c-5aa798b7cfb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3b3a526943646dcba7bb1806601696b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:56:24.877176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d3062354-10a6-4712-809c-5aa798b7cfb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3b3a526943646dcba7bb1806601696b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:56:24.898290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d3062354-10a6-4712-809c-5aa798b7cfb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3b3a526943646dcba7bb1806601696b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:56:24.904079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d3062354-10a6-4712-809c-5aa798b7cfb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3b3a526943646dcba7bb1806601696b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:56:24.940037 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1202/2415] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:56:24 2022] DELETE /identity/v3/users/d3b3a526943646dcba7bb1806601696b => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:56:25.311822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c982790-403d-4e18-a9c8-67476fb83ff2 service swift] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:25.330369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c982790-403d-4e18-a9c8-67476fb83ff2 service swift] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:25.465237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07098ada-10cc-4d60-94b4-4454120249a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:25.466805 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07098ada-10cc-4d60-94b4-4454120249a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:25.467308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07098ada-10cc-4d60-94b4-4454120249a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:25.467811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07098ada-10cc-4d60-94b4-4454120249a6 admin admin] PATH_INFO: `/v3/projects/a84353cfe41640e3b13d561b615e1142` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:25.470288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07098ada-10cc-4d60-94b4-4454120249a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a84353cfe41640e3b13d561b615e1142)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:25.474302 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07098ada-10cc-4d60-94b4-4454120249a6 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:25.593845 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:56:25.632713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-07098ada-10cc-4d60-94b4-4454120249a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a84353cfe41640e3b13d561b615e1142', 'request_id': 'req-07098ada-10cc-4d60-94b4-4454120249a6'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:56:25.654814 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-07098ada-10cc-4d60-94b4-4454120249a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a84353cfe41640e3b13d561b615e1142', 'request_id': 'req-07098ada-10cc-4d60-94b4-4454120249a6'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:56:25.698571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-07098ada-10cc-4d60-94b4-4454120249a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a84353cfe41640e3b13d561b615e1142', 'request_id': 'req-07098ada-10cc-4d60-94b4-4454120249a6'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:56:25.743909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-07098ada-10cc-4d60-94b4-4454120249a6 admin admin] The token cache is being invalidate because project a84353cfe41640e3b13d561b615e1142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:56:25.744631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-07098ada-10cc-4d60-94b4-4454120249a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:56:25.746740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-07098ada-10cc-4d60-94b4-4454120249a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:56:25.752576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1214/2416] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:56:25 2022] DELETE /identity/v3/projects/a84353cfe41640e3b13d561b615e1142 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:56:25.759964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7ccb1-1005-4a02-b5ce-d4ae74900421 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:25.760605 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7ccb1-1005-4a02-b5ce-d4ae74900421 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:25.761157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb7ccb1-1005-4a02-b5ce-d4ae74900421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:25.820764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-0cb7ccb1-1005-4a02-b5ce-d4ae74900421 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:56:25.828012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0cb7ccb1-1005-4a02-b5ce-d4ae74900421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:25.978290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:56:26.020682 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1203/2417] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:56:26.033605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6541d6-5c5d-4561-a462-c77f070d1bc1 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:26.034248 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6541d6-5c5d-4561-a462-c77f070d1bc1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:26.034840 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6541d6-5c5d-4561-a462-c77f070d1bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:26.075605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-ce6541d6-5c5d-4561-a462-c77f070d1bc1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:56:26.077711 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ce6541d6-5c5d-4561-a462-c77f070d1bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:26.103772 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1215/2418] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:56:26.105619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7ccb1-1005-4a02-b5ce-d4ae74900421 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:26.120476 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb7ccb1-1005-4a02-b5ce-d4ae74900421 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:26.124944 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3bc861-8496-415c-b69c-7dea88088764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:26.125845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3bc861-8496-415c-b69c-7dea88088764 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:26.126430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3bc861-8496-415c-b69c-7dea88088764 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:26.126849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3bc861-8496-415c-b69c-7dea88088764 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:26.128005 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3bc861-8496-415c-b69c-7dea88088764 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 17:56:26.128178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3bc861-8496-415c-b69c-7dea88088764 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:26.130142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3bc861-8496-415c-b69c-7dea88088764 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:26.147645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1204/2419] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 17:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 17:56:26.152901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6541d6-5c5d-4561-a462-c77f070d1bc1 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:26.164923 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6541d6-5c5d-4561-a462-c77f070d1bc1 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:26.170650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f71d3-8f96-4670-b82f-a11a31307fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:26.171985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f71d3-8f96-4670-b82f-a11a31307fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:26.171985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f71d3-8f96-4670-b82f-a11a31307fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:26.171985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650f71d3-8f96-4670-b82f-a11a31307fc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:26.172751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f71d3-8f96-4670-b82f-a11a31307fc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:26.175031 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650f71d3-8f96-4670-b82f-a11a31307fc4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:26.242537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1216/2420] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 17:56:26 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:56:26.244477 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3bc861-8496-415c-b69c-7dea88088764 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:26.306519 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3bc861-8496-415c-b69c-7dea88088764 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:26.474762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657a88e-6cdb-4dfe-b0c3-51756c967853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:26.477058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657a88e-6cdb-4dfe-b0c3-51756c967853 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:26.477717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657a88e-6cdb-4dfe-b0c3-51756c967853 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:26.478305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7657a88e-6cdb-4dfe-b0c3-51756c967853 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:26.481387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657a88e-6cdb-4dfe-b0c3-51756c967853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:26.484556 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7657a88e-6cdb-4dfe-b0c3-51756c967853 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:26.524966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1205/2421] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 17:56:26 2022] POST /identity/v3/users => generated 288 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 17:56:26.529794 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f71d3-8f96-4670-b82f-a11a31307fc4 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:26.544642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650f71d3-8f96-4670-b82f-a11a31307fc4 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:26.548623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4ea8-39d9-4ae1-bb68-943804e33c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:26.549281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4ea8-39d9-4ae1-bb68-943804e33c93 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:26.549698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4ea8-39d9-4ae1-bb68-943804e33c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:26.549894 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa4ea8-39d9-4ae1-bb68-943804e33c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:26.550597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa4ea8-39d9-4ae1-bb68-943804e33c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:26.552323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aa4ea8-39d9-4ae1-bb68-943804e33c93 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:26.578975 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657a88e-6cdb-4dfe-b0c3-51756c967853 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:26.579622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1217/2422] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:56:26.595328 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7657a88e-6cdb-4dfe-b0c3-51756c967853 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:26.608200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6259e-f2ab-445f-979e-42e45d5fb42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:26.609149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6259e-f2ab-445f-979e-42e45d5fb42d admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:26.609543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6259e-f2ab-445f-979e-42e45d5fb42d admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:26.609892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6259e-f2ab-445f-979e-42e45d5fb42d admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:26.610767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6259e-f2ab-445f-979e-42e45d5fb42d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:26.612498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6259e-f2ab-445f-979e-42e45d5fb42d admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:26.636487 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1206/2423] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 17:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 17:56:26.640766 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4ea8-39d9-4ae1-bb68-943804e33c93 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:26.656045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa4ea8-39d9-4ae1-bb68-943804e33c93 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:26.664707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f06242-48cc-4751-a5cd-1c1d011cd516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:26.666319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f06242-48cc-4751-a5cd-1c1d011cd516 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:26.667356 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f06242-48cc-4751-a5cd-1c1d011cd516 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:26.667872 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f06242-48cc-4751-a5cd-1c1d011cd516 admin admin] PATH_INFO: `/v3/projects/052d2b0826d04dd895c3621cf6d87e17/users/353609092f9b4a378478169ad8b90503/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:26.700885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f06242-48cc-4751-a5cd-1c1d011cd516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=052d2b0826d04dd895c3621cf6d87e17, user_id=353609092f9b4a378478169ad8b90503, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:26.706860 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f06242-48cc-4751-a5cd-1c1d011cd516 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:26.753004 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1218/2424] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 17:56:26 2022] PUT /identity/v3/projects/052d2b0826d04dd895c3621cf6d87e17/users/353609092f9b4a378478169ad8b90503/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:56:26.772112 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69dbb0-5a22-44f0-a2cd-abae37a56038 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:26.772830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69dbb0-5a22-44f0-a2cd-abae37a56038 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:26.773308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa69dbb0-5a22-44f0-a2cd-abae37a56038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:26.824662 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-fa69dbb0-5a22-44f0-a2cd-abae37a56038 None None] MFA Rules not processed for user `353609092f9b4a378478169ad8b90503`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:56:26.826817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-fa69dbb0-5a22-44f0-a2cd-abae37a56038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:27.002925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1207/2425] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:56:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:56:27.035153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f06242-48cc-4751-a5cd-1c1d011cd516 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:27.038191 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-81f06242-48cc-4751-a5cd-1c1d011cd516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:27.155029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f06242-48cc-4751-a5cd-1c1d011cd516 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:27.164887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e60abe-d5e3-48ba-9c38-de014b4fc719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:27.165567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e60abe-d5e3-48ba-9c38-de014b4fc719 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:27.165861 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e60abe-d5e3-48ba-9c38-de014b4fc719 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:27.166139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e60abe-d5e3-48ba-9c38-de014b4fc719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:27.191476 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e60abe-d5e3-48ba-9c38-de014b4fc719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:27.203876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e60abe-d5e3-48ba-9c38-de014b4fc719 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:27.217270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1219/2426] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:56:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:56:28.922944 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b16b44-f9e5-4fdb-8a43-58ff23b05a02 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:28.923835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b16b44-f9e5-4fdb-8a43-58ff23b05a02 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:28.924044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b16b44-f9e5-4fdb-8a43-58ff23b05a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:28.963022 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-a0b16b44-f9e5-4fdb-8a43-58ff23b05a02 None None] MFA Rules not processed for user `353609092f9b4a378478169ad8b90503`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:56:28.964451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a0b16b44-f9e5-4fdb-8a43-58ff23b05a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:28.977815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1208/2427] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:56:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:56:37.210789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:56:38.956394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:56:48.801264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e60abe-d5e3-48ba-9c38-de014b4fc719 service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:48.802945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a5e60abe-d5e3-48ba-9c38-de014b4fc719 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:48.868427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e60abe-d5e3-48ba-9c38-de014b4fc719 service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:48.875955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5d0ff5-3fc6-46fa-b2b5-02eaeb42a828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:48.877327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d0ff5-3fc6-46fa-b2b5-02eaeb42a828 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:48.879395 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d0ff5-3fc6-46fa-b2b5-02eaeb42a828 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:48.879395 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5d0ff5-3fc6-46fa-b2b5-02eaeb42a828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:48.881429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-cb5d0ff5-3fc6-46fa-b2b5-02eaeb42a828 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:48.975108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d0ff5-3fc6-46fa-b2b5-02eaeb42a828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:48.976656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5d0ff5-3fc6-46fa-b2b5-02eaeb42a828 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:48.989070 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1220/2428] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:56:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:56:50.752684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b16b44-f9e5-4fdb-8a43-58ff23b05a02 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:56:50.764141 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b16b44-f9e5-4fdb-8a43-58ff23b05a02 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:56:50.767857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f70540-3baf-4a6b-8a87-cc0c8a80442a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:56:50.768602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f70540-3baf-4a6b-8a87-cc0c8a80442a service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:56:50.768891 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f70540-3baf-4a6b-8a87-cc0c8a80442a service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:56:50.769171 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f70540-3baf-4a6b-8a87-cc0c8a80442a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:56:50.771083 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-51f70540-3baf-4a6b-8a87-cc0c8a80442a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:56:50.785363 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f70540-3baf-4a6b-8a87-cc0c8a80442a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:56:50.787069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f70540-3baf-4a6b-8a87-cc0c8a80442a service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:56:50.798801 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1209/2429] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:56:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:56:58.983732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=54 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:57:00.793615 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=6 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:57:03.343293 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cdc4b-e002-41b5-ac35-07e0cb347aa9 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:03.343791 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cdc4b-e002-41b5-ac35-07e0cb347aa9 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:03.343995 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9cdc4b-e002-41b5-ac35-07e0cb347aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:03.363203 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-0d9cdc4b-e002-41b5-ac35-07e0cb347aa9 None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:57:03.364383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0d9cdc4b-e002-41b5-ac35-07e0cb347aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:03.425013 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1221/2430] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 17:57:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:57:03.450716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f70540-3baf-4a6b-8a87-cc0c8a80442a service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:03.452747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-51f70540-3baf-4a6b-8a87-cc0c8a80442a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:03.523958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f70540-3baf-4a6b-8a87-cc0c8a80442a service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:03.529002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd02b2-669d-4c40-9035-c1b432541696 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:03.529767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd02b2-669d-4c40-9035-c1b432541696 service swift] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:03.530058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd02b2-669d-4c40-9035-c1b432541696 service swift] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:03.530341 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd02b2-669d-4c40-9035-c1b432541696 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:03.539159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd02b2-669d-4c40-9035-c1b432541696 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:03.540534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd02b2-669d-4c40-9035-c1b432541696 service swift] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:03.549134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1210/2431] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 17:57:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 17:57:04.106745 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b0de-d819-4577-b1bb-af8658721b9b None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:04.107056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b0de-d819-4577-b1bb-af8658721b9b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:04.107370 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6610b0de-d819-4577-b1bb-af8658721b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:04.130150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-6610b0de-d819-4577-b1bb-af8658721b9b None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:57:04.131515 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6610b0de-d819-4577-b1bb-af8658721b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:04.139336 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1222/2432] 10.222.0.64 () {58 vars in 967 bytes} [Thu Jul 14 17:57:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:57:04.154256 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd02b2-669d-4c40-9035-c1b432541696 service swift] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:04.166446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd02b2-669d-4c40-9035-c1b432541696 service swift] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:04.175224 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e395b38-049f-45c9-a643-8d6ba6aab30c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:04.176354 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e395b38-049f-45c9-a643-8d6ba6aab30c service swift] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:04.176666 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e395b38-049f-45c9-a643-8d6ba6aab30c service swift] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:04.177326 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e395b38-049f-45c9-a643-8d6ba6aab30c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:04.187689 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e395b38-049f-45c9-a643-8d6ba6aab30c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:04.189491 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e395b38-049f-45c9-a643-8d6ba6aab30c service swift] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:04.198730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1211/2433] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 17:57:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 17:57:04.410523 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8a0a2-7b84-48ba-8a58-2b454ebf8df1 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:04.411054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8a0a2-7b84-48ba-8a58-2b454ebf8df1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:04.411054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8a0a2-7b84-48ba-8a58-2b454ebf8df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:04.428994 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-eda8a0a2-7b84-48ba-8a58-2b454ebf8df1 None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:57:04.430193 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-eda8a0a2-7b84-48ba-8a58-2b454ebf8df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:04.438150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1223/2434] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 17:57:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:57:04.449984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e395b38-049f-45c9-a643-8d6ba6aab30c service swift] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:04.460863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e395b38-049f-45c9-a643-8d6ba6aab30c service swift] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:04.465814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ba9be-3cf9-49e1-b716-555aec8c9ba4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:04.466450 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ba9be-3cf9-49e1-b716-555aec8c9ba4 service swift] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:04.466741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ba9be-3cf9-49e1-b716-555aec8c9ba4 service swift] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:04.467029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ba9be-3cf9-49e1-b716-555aec8c9ba4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:04.475111 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ba9be-3cf9-49e1-b716-555aec8c9ba4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:04.476740 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ba9be-3cf9-49e1-b716-555aec8c9ba4 service swift] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:04.485479 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1212/2435] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 17:57:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 17:57:04.828669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3915622-8aec-4e7a-b8fc-f5528449414f None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:04.828953 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3915622-8aec-4e7a-b8fc-f5528449414f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:04.829240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3915622-8aec-4e7a-b8fc-f5528449414f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:04.853205 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-a3915622-8aec-4e7a-b8fc-f5528449414f None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:57:04.854441 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a3915622-8aec-4e7a-b8fc-f5528449414f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:04.863264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1224/2436] 10.222.0.64 () {58 vars in 967 bytes} [Thu Jul 14 17:57:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:57:04.877008 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ba9be-3cf9-49e1-b716-555aec8c9ba4 service swift] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:04.888400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ba9be-3cf9-49e1-b716-555aec8c9ba4 service swift] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:04.894217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21911ef6-29a0-4948-9ba8-a813ac73ae7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:04.894893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21911ef6-29a0-4948-9ba8-a813ac73ae7e service swift] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:04.895182 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21911ef6-29a0-4948-9ba8-a813ac73ae7e service swift] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:04.895464 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21911ef6-29a0-4948-9ba8-a813ac73ae7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:04.906314 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21911ef6-29a0-4948-9ba8-a813ac73ae7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:04.907835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21911ef6-29a0-4948-9ba8-a813ac73ae7e service swift] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:04.917203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1213/2437] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 17:57:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 17:57:06.739751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3915622-8aec-4e7a-b8fc-f5528449414f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:06.741383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a3915622-8aec-4e7a-b8fc-f5528449414f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:06.758718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3915622-8aec-4e7a-b8fc-f5528449414f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:06.767639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd88845-2f76-4543-9d67-bb7478bf6437 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:06.768099 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd88845-2f76-4543-9d67-bb7478bf6437 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:06.768366 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd88845-2f76-4543-9d67-bb7478bf6437 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:06.768865 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd88845-2f76-4543-9d67-bb7478bf6437 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:06.770570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6dd88845-2f76-4543-9d67-bb7478bf6437 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:06.833120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd88845-2f76-4543-9d67-bb7478bf6437 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:06.834881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd88845-2f76-4543-9d67-bb7478bf6437 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:06.844922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1225/2438] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:57:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:57:07.366148 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21911ef6-29a0-4948-9ba8-a813ac73ae7e service swift] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:07.430355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21911ef6-29a0-4948-9ba8-a813ac73ae7e service swift] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:07.435009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737e340-8ade-40e9-97c0-033a4520a030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:07.435890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737e340-8ade-40e9-97c0-033a4520a030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:07.436228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737e340-8ade-40e9-97c0-033a4520a030 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:07.436507 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d737e340-8ade-40e9-97c0-033a4520a030 admin admin] PATH_INFO: `/v3/users/353609092f9b4a378478169ad8b90503` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:07.437729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737e340-8ade-40e9-97c0-033a4520a030 admin admin] RBAC: Authorizing `identity:delete_user(user_id=353609092f9b4a378478169ad8b90503)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:07.439650 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d737e340-8ade-40e9-97c0-033a4520a030 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:07.490171 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d737e340-8ade-40e9-97c0-033a4520a030 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '353609092f9b4a378478169ad8b90503'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:57:07.497221 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:57:07.510747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d737e340-8ade-40e9-97c0-033a4520a030 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '353609092f9b4a378478169ad8b90503'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:57:07.511983 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d737e340-8ade-40e9-97c0-033a4520a030 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '353609092f9b4a378478169ad8b90503'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:57:07.521517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d737e340-8ade-40e9-97c0-033a4520a030 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '353609092f9b4a378478169ad8b90503'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:57:07.522162 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d737e340-8ade-40e9-97c0-033a4520a030 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '353609092f9b4a378478169ad8b90503'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:57:07.537766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1214/2439] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 17:57:07 2022] DELETE /identity/v3/users/353609092f9b4a378478169ad8b90503 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 17:57:07.542589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1cb9-c1d6-47c9-852b-afac94cc1fd9 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:07.543014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1cb9-c1d6-47c9-852b-afac94cc1fd9 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:07.543388 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc1cb9-c1d6-47c9-852b-afac94cc1fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:07.562296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-8cbc1cb9-c1d6-47c9-852b-afac94cc1fd9 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:57:07.563915 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8cbc1cb9-c1d6-47c9-852b-afac94cc1fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:07.634628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1226/2440] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:57:07.646895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737e340-8ade-40e9-97c0-033a4520a030 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:07.648394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d737e340-8ade-40e9-97c0-033a4520a030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:07.713242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d737e340-8ade-40e9-97c0-033a4520a030 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:07.717116 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3952e25-413d-494b-a011-0f0482001bca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:07.718415 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3952e25-413d-494b-a011-0f0482001bca service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:07.718700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3952e25-413d-494b-a011-0f0482001bca service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:07.718973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3952e25-413d-494b-a011-0f0482001bca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:07.729270 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3952e25-413d-494b-a011-0f0482001bca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:07.730748 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3952e25-413d-494b-a011-0f0482001bca service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:07.741067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1215/2441] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:57:07.928555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc1cb9-c1d6-47c9-852b-afac94cc1fd9 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:07.930275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8cbc1cb9-c1d6-47c9-852b-afac94cc1fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:07.943733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc1cb9-c1d6-47c9-852b-afac94cc1fd9 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:07.949457 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:07.950342 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:07.950631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:07.950913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] PATH_INFO: `/v3/projects/052d2b0826d04dd895c3621cf6d87e17` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:07.953584 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] RBAC: Authorizing `identity:delete_project(project_id=052d2b0826d04dd895c3621cf6d87e17)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:07.954774 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:07.977753 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:57:08.005144 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '052d2b0826d04dd895c3621cf6d87e17', 'request_id': 'req-0f928ee5-c6a0-4cb5-b836-ff9452788eac'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:57:08.016805 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '052d2b0826d04dd895c3621cf6d87e17', 'request_id': 'req-0f928ee5-c6a0-4cb5-b836-ff9452788eac'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:57:08.024386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '052d2b0826d04dd895c3621cf6d87e17', 'request_id': 'req-0f928ee5-c6a0-4cb5-b836-ff9452788eac'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:57:08.035534 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] The token cache is being invalidate because project 052d2b0826d04dd895c3621cf6d87e17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 17:57:08.035941 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:57:08.036518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 17:57:08.038213 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1227/2442] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 17:57:07 2022] DELETE /identity/v3/projects/052d2b0826d04dd895c3621cf6d87e17 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 17:57:08.668543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3952e25-413d-494b-a011-0f0482001bca service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:08.670057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a3952e25-413d-494b-a011-0f0482001bca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:08.753607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3952e25-413d-494b-a011-0f0482001bca service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:08.823870 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355e18b-64b9-40b3-b4a1-f92e07a58eaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:08.824567 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355e18b-64b9-40b3-b4a1-f92e07a58eaa service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:08.824857 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355e18b-64b9-40b3-b4a1-f92e07a58eaa service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:08.825140 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355e18b-64b9-40b3-b4a1-f92e07a58eaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:08.827717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7355e18b-64b9-40b3-b4a1-f92e07a58eaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:08.834492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:57:08.928270 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355e18b-64b9-40b3-b4a1-f92e07a58eaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:08.931421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7355e18b-64b9-40b3-b4a1-f92e07a58eaa service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:08.946444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1216/2443] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:57:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:57:11.723787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:11.726192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:11.797198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f928ee5-c6a0-4cb5-b836-ff9452788eac admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:11.801118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60f709-42e7-4a64-bd88-fd671bbeb41a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:11.801776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60f709-42e7-4a64-bd88-fd671bbeb41a service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:11.802063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60f709-42e7-4a64-bd88-fd671bbeb41a service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:11.802343 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60f709-42e7-4a64-bd88-fd671bbeb41a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:11.804155 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8a60f709-42e7-4a64-bd88-fd671bbeb41a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:11.880900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a60f709-42e7-4a64-bd88-fd671bbeb41a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:11.882546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a60f709-42e7-4a64-bd88-fd671bbeb41a service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:11.893862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1228/2444] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:57:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:57:13.214695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f8b9a-a628-4f37-90ba-d177f945bf21 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:13.215202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f8b9a-a628-4f37-90ba-d177f945bf21 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:13.215308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f8b9a-a628-4f37-90ba-d177f945bf21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:13.234996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-016f8b9a-a628-4f37-90ba-d177f945bf21 None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:57:13.236353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-016f8b9a-a628-4f37-90ba-d177f945bf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:13.318691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1217/2445] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 17:57:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:57:13.325598 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60f709-42e7-4a64-bd88-fd671bbeb41a service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:13.327877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8a60f709-42e7-4a64-bd88-fd671bbeb41a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:13.396732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:57:13.414579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60f709-42e7-4a64-bd88-fd671bbeb41a service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:13.421322 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd8546-3a2c-4628-ac8d-62c20356b209 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:13.421951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd8546-3a2c-4628-ac8d-62c20356b209 service swift] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:13.422249 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd8546-3a2c-4628-ac8d-62c20356b209 service swift] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:13.422571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd8546-3a2c-4628-ac8d-62c20356b209 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:13.431420 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd8546-3a2c-4628-ac8d-62c20356b209 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:13.433153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd8546-3a2c-4628-ac8d-62c20356b209 service swift] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:13.443455 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1229/2446] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 17:57:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 17:57:14.695544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f8b9a-a628-4f37-90ba-d177f945bf21 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:14.697703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-016f8b9a-a628-4f37-90ba-d177f945bf21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:14.712783 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f8b9a-a628-4f37-90ba-d177f945bf21 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:14.717243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b449e7-e5b6-4932-890e-428690bbceb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:14.718833 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b449e7-e5b6-4932-890e-428690bbceb1 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:14.719528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b449e7-e5b6-4932-890e-428690bbceb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:14.720004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b449e7-e5b6-4932-890e-428690bbceb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:14.722114 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-97b449e7-e5b6-4932-890e-428690bbceb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:14.735902 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b449e7-e5b6-4932-890e-428690bbceb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:14.739142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b449e7-e5b6-4932-890e-428690bbceb1 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:14.762877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1218/2447] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:57:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:57:23.441310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=6 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:57:24.745480 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=55 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:57:33.263759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd8546-3a2c-4628-ac8d-62c20356b209 service swift] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:33.273884 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd8546-3a2c-4628-ac8d-62c20356b209 service swift] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:33.277739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f615034-ca6f-4118-88b7-753b8f769017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:33.278404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f615034-ca6f-4118-88b7-753b8f769017 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:33.278692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f615034-ca6f-4118-88b7-753b8f769017 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:33.278969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f615034-ca6f-4118-88b7-753b8f769017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:33.280876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5f615034-ca6f-4118-88b7-753b8f769017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:33.291621 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f615034-ca6f-4118-88b7-753b8f769017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:33.293102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f615034-ca6f-4118-88b7-753b8f769017 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:33.302615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1230/2448] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:57:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:57:33.423020 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aee427-cf3d-4e6d-a235-8973718658ea None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:33.423686 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aee427-cf3d-4e6d-a235-8973718658ea None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:33.424700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aee427-cf3d-4e6d-a235-8973718658ea None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:33.427362 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1219/2449] 10.222.0.64 () {54 vars in 921 bytes} [Thu Jul 14 17:57:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:57:33.434013 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484751d-8c37-42a3-9294-255b425f72dc None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:33.434300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484751d-8c37-42a3-9294-255b425f72dc None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:33.434583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6484751d-8c37-42a3-9294-255b425f72dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:33.454922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-6484751d-8c37-42a3-9294-255b425f72dc None None] MFA Rules not processed for user `6423f54152064b909d03c0908d636993`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:57:33.456216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6484751d-8c37-42a3-9294-255b425f72dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:33.466766 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1231/2450] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Jul 14 17:57:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:57:33.480279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aee427-cf3d-4e6d-a235-8973718658ea None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:33.482685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d6aee427-cf3d-4e6d-a235-8973718658ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:33.560872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aee427-cf3d-4e6d-a235-8973718658ea None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:33.564536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe5832-881a-4a2c-b991-033e92f47494 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2b4bf2aa1744a46b6d2a1b5a136a0e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:33.565211 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe5832-881a-4a2c-b991-033e92f47494 service glance] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:33.565487 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe5832-881a-4a2c-b991-033e92f47494 service glance] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:33.565760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe5832-881a-4a2c-b991-033e92f47494 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:33.577570 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe5832-881a-4a2c-b991-033e92f47494 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:33.579019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe5832-881a-4a2c-b991-033e92f47494 service glance] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:33.591503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1220/2451] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:57:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:57:33.664720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82718b-b251-4afd-8bd9-6b88775110f0 None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:33.665009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82718b-b251-4afd-8bd9-6b88775110f0 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:33.665289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da82718b-b251-4afd-8bd9-6b88775110f0 None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:33.666707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1232/2452] 10.222.0.64 () {54 vars in 921 bytes} [Thu Jul 14 17:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 17:57:33.672432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761d65d-0f27-444c-8e41-ae8fe7f92634 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:33.672845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761d65d-0f27-444c-8e41-ae8fe7f92634 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:33.673151 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761d65d-0f27-444c-8e41-ae8fe7f92634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:33.691844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-6761d65d-0f27-444c-8e41-ae8fe7f92634 None None] MFA Rules not processed for user `a5adec9924444e29b8964f7969452592`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:57:33.693153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6761d65d-0f27-444c-8e41-ae8fe7f92634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:33.700473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1221/2453] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Jul 14 17:57:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:57:33.713651 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82718b-b251-4afd-8bd9-6b88775110f0 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:33.725782 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da82718b-b251-4afd-8bd9-6b88775110f0 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:33.729380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1dd372-d122-4cb1-b412-3250f18b69fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:33.730032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dd372-d122-4cb1-b412-3250f18b69fd service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:33.730319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dd372-d122-4cb1-b412-3250f18b69fd service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:33.730599 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dd372-d122-4cb1-b412-3250f18b69fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:33.738468 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1dd372-d122-4cb1-b412-3250f18b69fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:33.739912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1dd372-d122-4cb1-b412-3250f18b69fd service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:33.748180 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1233/2454] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:57:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:57:33.836110 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19332126-3d89-4923-acb4-1367c1097fc2 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:33.836397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19332126-3d89-4923-acb4-1367c1097fc2 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:33.836678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19332126-3d89-4923-acb4-1367c1097fc2 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:33.838417 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1222/2455] 10.222.0.64 () {54 vars in 921 bytes} [Thu Jul 14 17:57:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 17:57:33.844913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc5ab4-213f-496e-9b36-45b52c89b5bb None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:33.845209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc5ab4-213f-496e-9b36-45b52c89b5bb None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:33.845494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc5ab4-213f-496e-9b36-45b52c89b5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:33.864659 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-65dc5ab4-213f-496e-9b36-45b52c89b5bb None None] MFA Rules not processed for user `6423f54152064b909d03c0908d636993`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:57:33.865885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-65dc5ab4-213f-496e-9b36-45b52c89b5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:33.876273 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1234/2456] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Jul 14 17:57:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:57:33.887453 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19332126-3d89-4923-acb4-1367c1097fc2 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:33.899382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19332126-3d89-4923-acb4-1367c1097fc2 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:33.903200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df301ddc-8a47-4bec-b1ba-1a0181542ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:33.903875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df301ddc-8a47-4bec-b1ba-1a0181542ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:33.904194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df301ddc-8a47-4bec-b1ba-1a0181542ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:33.904492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df301ddc-8a47-4bec-b1ba-1a0181542ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:33.915842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df301ddc-8a47-4bec-b1ba-1a0181542ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:33.917303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df301ddc-8a47-4bec-b1ba-1a0181542ca1 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:33.929339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1223/2457] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:57:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:57:33.993481 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc5ab4-213f-496e-9b36-45b52c89b5bb None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:33.994870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-65dc5ab4-213f-496e-9b36-45b52c89b5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:34.065296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc5ab4-213f-496e-9b36-45b52c89b5bb None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:34.070442 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01390820-d34d-4099-9500-0a1f55b607cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:34.071115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01390820-d34d-4099-9500-0a1f55b607cd service placement] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:34.071406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01390820-d34d-4099-9500-0a1f55b607cd service placement] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:34.071725 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01390820-d34d-4099-9500-0a1f55b607cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:34.073554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-01390820-d34d-4099-9500-0a1f55b607cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:34.086402 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01390820-d34d-4099-9500-0a1f55b607cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:34.088293 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01390820-d34d-4099-9500-0a1f55b607cd service placement] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:34.100644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1235/2458] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:57:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:57:36.887795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06b8c5-4a7c-46be-bb05-869125e7bf60 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:36.889306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06b8c5-4a7c-46be-bb05-869125e7bf60 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:36.889306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06b8c5-4a7c-46be-bb05-869125e7bf60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:36.910966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-9b06b8c5-4a7c-46be-bb05-869125e7bf60 None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:57:36.912322 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9b06b8c5-4a7c-46be-bb05-869125e7bf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:57:36.919577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1224/2459] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 17:57:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:57:36.933095 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01390820-d34d-4099-9500-0a1f55b607cd service placement] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:57:36.945165 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01390820-d34d-4099-9500-0a1f55b607cd service placement] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:57:36.952689 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a9910-d893-4138-ba79-785e6ca0987a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:57:36.953618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a9910-d893-4138-ba79-785e6ca0987a service swift] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:57:36.954110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a9910-d893-4138-ba79-785e6ca0987a service swift] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:57:36.954592 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a9910-d893-4138-ba79-785e6ca0987a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:57:36.964088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a9910-d893-4138-ba79-785e6ca0987a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:57:36.965694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a9910-d893-4138-ba79-785e6ca0987a service swift] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:57:36.975069 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1236/2460] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 17:57:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 17:57:46.905470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=38 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:57:46.971290 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=56 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:58:06.832460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06b8c5-4a7c-46be-bb05-869125e7bf60 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:58:06.834019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9b06b8c5-4a7c-46be-bb05-869125e7bf60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:58:06.853136 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06b8c5-4a7c-46be-bb05-869125e7bf60 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:58:06.864373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb717e9-b562-4b54-9a39-c1a85fe60514 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:58:06.865561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb717e9-b562-4b54-9a39-c1a85fe60514 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:58:06.865846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb717e9-b562-4b54-9a39-c1a85fe60514 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:58:06.866126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb717e9-b562-4b54-9a39-c1a85fe60514 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:58:06.868059 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ceb717e9-b562-4b54-9a39-c1a85fe60514 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:58:06.885238 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb717e9-b562-4b54-9a39-c1a85fe60514 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:58:06.887718 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb717e9-b562-4b54-9a39-c1a85fe60514 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:58:06.900583 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1225/2461] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:58:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:58:10.592350 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15ca68-5bb0-41ea-8b24-6ad22c12b485 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:58:10.593154 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15ca68-5bb0-41ea-8b24-6ad22c12b485 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:58:10.593154 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15ca68-5bb0-41ea-8b24-6ad22c12b485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:58:10.633921 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-6d15ca68-5bb0-41ea-8b24-6ad22c12b485 None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:58:10.636328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6d15ca68-5bb0-41ea-8b24-6ad22c12b485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:58:10.684143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1237/2462] 10.222.0.64 () {58 vars in 967 bytes} [Thu Jul 14 17:58:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:58:10.709757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb717e9-b562-4b54-9a39-c1a85fe60514 service nova] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:58:10.764670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb717e9-b562-4b54-9a39-c1a85fe60514 service nova] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:58:10.770335 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8d3ee4-6604-4b23-9fd4-8c195fbbd081 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:58:10.771143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d3ee4-6604-4b23-9fd4-8c195fbbd081 service swift] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:58:10.771670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d3ee4-6604-4b23-9fd4-8c195fbbd081 service swift] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:58:10.772045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8d3ee4-6604-4b23-9fd4-8c195fbbd081 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:58:10.783423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d3ee4-6604-4b23-9fd4-8c195fbbd081 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:58:10.785424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8d3ee4-6604-4b23-9fd4-8c195fbbd081 service swift] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:58:10.795849 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1226/2463] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 17:58:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 17:58:20.681448 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=20 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:58:20.792167 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=26 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:58:33.351713 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15ca68-5bb0-41ea-8b24-6ad22c12b485 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:58:33.371132 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15ca68-5bb0-41ea-8b24-6ad22c12b485 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:58:33.378332 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f19ea8-3378-4a14-ae9a-b72414dbfc0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:58:33.379154 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f19ea8-3378-4a14-ae9a-b72414dbfc0f service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:58:33.379539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f19ea8-3378-4a14-ae9a-b72414dbfc0f service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:58:33.379853 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f19ea8-3378-4a14-ae9a-b72414dbfc0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:58:33.382724 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e8f19ea8-3378-4a14-ae9a-b72414dbfc0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:58:33.455800 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f19ea8-3378-4a14-ae9a-b72414dbfc0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:58:33.457450 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f19ea8-3378-4a14-ae9a-b72414dbfc0f service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:58:33.467090 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1238/2464] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:58:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:58:43.462353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=28 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:58:57.164346 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d64fc4-248a-4f7f-af07-c6dbcfd10df2 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:58:57.164346 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d64fc4-248a-4f7f-af07-c6dbcfd10df2 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:58:57.164346 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d64fc4-248a-4f7f-af07-c6dbcfd10df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:58:57.184411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-78d64fc4-248a-4f7f-af07-c6dbcfd10df2 None None] MFA Rules not processed for user `52c794c11ab74f9ca9bad75857043a62`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:58:57.185637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-78d64fc4-248a-4f7f-af07-c6dbcfd10df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:58:57.193441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1227/2465] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:58:57 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 17:59:03.687274 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f19ea8-3378-4a14-ae9a-b72414dbfc0f service nova] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:59:03.689573 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e8f19ea8-3378-4a14-ae9a-b72414dbfc0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:59:03.707272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f19ea8-3378-4a14-ae9a-b72414dbfc0f service nova] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:59:03.714891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac4135-6ee0-4d1a-a26e-a0d85644d207 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:59:03.716706 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac4135-6ee0-4d1a-a26e-a0d85644d207 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:59:03.716706 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac4135-6ee0-4d1a-a26e-a0d85644d207 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:59:03.717102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac4135-6ee0-4d1a-a26e-a0d85644d207 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:59:03.721632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a4ac4135-6ee0-4d1a-a26e-a0d85644d207 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:59:03.739304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac4135-6ee0-4d1a-a26e-a0d85644d207 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:59:03.739658 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac4135-6ee0-4d1a-a26e-a0d85644d207 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:59:03.759219 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1239/2466] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:59:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 17:59:07.178254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:59:13.753215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=6 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:59:37.735878 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d64fc4-248a-4f7f-af07-c6dbcfd10df2 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:59:37.750643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d64fc4-248a-4f7f-af07-c6dbcfd10df2 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:59:37.756621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1f00c-8c43-4260-b5a1-a54a2e3ff8fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:59:37.757812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1f00c-8c43-4260-b5a1-a54a2e3ff8fa service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:59:37.758100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1f00c-8c43-4260-b5a1-a54a2e3ff8fa service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:59:37.758375 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1f00c-8c43-4260-b5a1-a54a2e3ff8fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:59:37.760520 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7ac1f00c-8c43-4260-b5a1-a54a2e3ff8fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:59:37.778900 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1f00c-8c43-4260-b5a1-a54a2e3ff8fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:59:37.780606 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1f00c-8c43-4260-b5a1-a54a2e3ff8fa service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:59:37.792697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1228/2467] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:59:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:59:41.041381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caabde-d01a-451a-97b7-b518061f8448 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:59:41.041916 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caabde-d01a-451a-97b7-b518061f8448 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:59:41.041916 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2caabde-d01a-451a-97b7-b518061f8448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:59:41.059263 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b2caabde-d01a-451a-97b7-b518061f8448 None None] MFA Rules not processed for user `c4178d34dc114f46af725c9dc362d224`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 17:59:41.060487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b2caabde-d01a-451a-97b7-b518061f8448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:59:41.134576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1240/2468] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 17:59:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 17:59:41.683528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1f00c-8c43-4260-b5a1-a54a2e3ff8fa service nova] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:59:41.704519 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1f00c-8c43-4260-b5a1-a54a2e3ff8fa service nova] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:59:41.709991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa7b51-7a7e-4a58-a426-bf561c27a842 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:59:41.711373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa7b51-7a7e-4a58-a426-bf561c27a842 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:59:41.711835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa7b51-7a7e-4a58-a426-bf561c27a842 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:59:41.712330 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa7b51-7a7e-4a58-a426-bf561c27a842 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:59:41.715194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ccaa7b51-7a7e-4a58-a426-bf561c27a842 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:59:41.740155 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa7b51-7a7e-4a58-a426-bf561c27a842 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:59:41.741680 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaa7b51-7a7e-4a58-a426-bf561c27a842 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:59:41.751310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1229/2469] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:59:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 17:59:51.126947 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=25 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:59:51.747247 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=21 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 17:59:52.993632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2caabde-d01a-451a-97b7-b518061f8448 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 17:59:53.034235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2caabde-d01a-451a-97b7-b518061f8448 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 17:59:53.041493 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81b9d7-dc9a-4397-bdff-a574ac92aa04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 17:59:53.042206 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81b9d7-dc9a-4397-bdff-a574ac92aa04 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 17:59:53.042617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81b9d7-dc9a-4397-bdff-a574ac92aa04 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 17:59:53.042863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81b9d7-dc9a-4397-bdff-a574ac92aa04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 17:59:53.045030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0d81b9d7-dc9a-4397-bdff-a574ac92aa04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 17:59:53.116326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81b9d7-dc9a-4397-bdff-a574ac92aa04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 17:59:53.117970 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d81b9d7-dc9a-4397-bdff-a574ac92aa04 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 17:59:53.129366 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1241/2470] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 17:59:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:00:03.123385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=35 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:00:40.900792 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa7b51-7a7e-4a58-a426-bf561c27a842 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:40.903295 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ccaa7b51-7a7e-4a58-a426-bf561c27a842 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:40.945908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaa7b51-7a7e-4a58-a426-bf561c27a842 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:40.951759 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e75ef-92ba-43ff-af21-061d45bc83f5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:40.952772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e75ef-92ba-43ff-af21-061d45bc83f5 service placement] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:40.953097 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e75ef-92ba-43ff-af21-061d45bc83f5 service placement] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:40.953616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e75ef-92ba-43ff-af21-061d45bc83f5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:40.957772 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-8d3e75ef-92ba-43ff-af21-061d45bc83f5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:40.975433 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e75ef-92ba-43ff-af21-061d45bc83f5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:40.978058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e75ef-92ba-43ff-af21-061d45bc83f5 service placement] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:40.992074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1230/2471] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:00:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:00:44.835895 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f438e-9949-434d-b6ea-83a1b7289b69 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:44.836433 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f438e-9949-434d-b6ea-83a1b7289b69 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:44.836533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f438e-9949-434d-b6ea-83a1b7289b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:44.853246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f83f438e-9949-434d-b6ea-83a1b7289b69 None None] MFA Rules not processed for user `84c38f6ff5af4ba79c80fb581baacb0b`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:00:44.854525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f83f438e-9949-434d-b6ea-83a1b7289b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:44.862300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1242/2472] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:00:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:00:50.986280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=15 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:00:51.934606 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e75ef-92ba-43ff-af21-061d45bc83f5 service placement] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:51.945457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e75ef-92ba-43ff-af21-061d45bc83f5 service placement] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:51.949143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376afc3-dc7d-45cc-ba9a-ac0f4d395550 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:51.949807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376afc3-dc7d-45cc-ba9a-ac0f4d395550 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:51.950090 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376afc3-dc7d-45cc-ba9a-ac0f4d395550 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:51.950368 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6376afc3-dc7d-45cc-ba9a-ac0f4d395550 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:51.952744 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6376afc3-dc7d-45cc-ba9a-ac0f4d395550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:52.039549 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6376afc3-dc7d-45cc-ba9a-ac0f4d395550 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:52.041585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6376afc3-dc7d-45cc-ba9a-ac0f4d395550 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:52.060217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1231/2473] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:00:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:00:54.848596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:00:55.732369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f438e-9949-434d-b6ea-83a1b7289b69 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:55.745697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f438e-9949-434d-b6ea-83a1b7289b69 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:55.751245 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f63612-6e49-4120-bd98-13e25bb2a4af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:55.752122 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f63612-6e49-4120-bd98-13e25bb2a4af service placement] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:55.752512 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f63612-6e49-4120-bd98-13e25bb2a4af service placement] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:55.752892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f63612-6e49-4120-bd98-13e25bb2a4af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:55.755707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-14f63612-6e49-4120-bd98-13e25bb2a4af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:55.768595 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f63612-6e49-4120-bd98-13e25bb2a4af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:55.770367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f63612-6e49-4120-bd98-13e25bb2a4af service placement] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:55.785232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1243/2474] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:00:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:00:55.807941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376afc3-dc7d-45cc-ba9a-ac0f4d395550 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:55.809778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6376afc3-dc7d-45cc-ba9a-ac0f4d395550 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:55.826465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6376afc3-dc7d-45cc-ba9a-ac0f4d395550 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:55.836432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe592f-e4a8-481e-b86e-2e270b18f7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:55.837507 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe592f-e4a8-481e-b86e-2e270b18f7e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:55.837860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe592f-e4a8-481e-b86e-2e270b18f7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:55.838227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe592f-e4a8-481e-b86e-2e270b18f7e6 admin admin] PATH_INFO: `/v3/users/52c794c11ab74f9ca9bad75857043a62` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:55.839563 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe592f-e4a8-481e-b86e-2e270b18f7e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c794c11ab74f9ca9bad75857043a62)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:55.841649 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe592f-e4a8-481e-b86e-2e270b18f7e6 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:55.900311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-debe592f-e4a8-481e-b86e-2e270b18f7e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c794c11ab74f9ca9bad75857043a62'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:55.913867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-debe592f-e4a8-481e-b86e-2e270b18f7e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c794c11ab74f9ca9bad75857043a62'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:55.914716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-debe592f-e4a8-481e-b86e-2e270b18f7e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c794c11ab74f9ca9bad75857043a62'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:55.924950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-debe592f-e4a8-481e-b86e-2e270b18f7e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c794c11ab74f9ca9bad75857043a62'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:55.925871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-debe592f-e4a8-481e-b86e-2e270b18f7e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c794c11ab74f9ca9bad75857043a62'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:55.947410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1232/2475] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:00:55 2022] DELETE /identity/v3/users/52c794c11ab74f9ca9bad75857043a62 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:00:56.163914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f63612-6e49-4120-bd98-13e25bb2a4af service placement] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:56.177428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f63612-6e49-4120-bd98-13e25bb2a4af service placement] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:56.262111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c88863-2bc6-4f11-9527-a5853720c495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:56.263066 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c88863-2bc6-4f11-9527-a5853720c495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:56.263422 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c88863-2bc6-4f11-9527-a5853720c495 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:56.263704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c88863-2bc6-4f11-9527-a5853720c495 admin admin] PATH_INFO: `/v3/projects/ab58d5c4c0194b58ad47dd5e4494389f` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:56.264931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c88863-2bc6-4f11-9527-a5853720c495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab58d5c4c0194b58ad47dd5e4494389f)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:56.267001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c88863-2bc6-4f11-9527-a5853720c495 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:56.326938 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b9c88863-2bc6-4f11-9527-a5853720c495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab58d5c4c0194b58ad47dd5e4494389f', 'request_id': 'req-b9c88863-2bc6-4f11-9527-a5853720c495'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:56.337214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b9c88863-2bc6-4f11-9527-a5853720c495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab58d5c4c0194b58ad47dd5e4494389f', 'request_id': 'req-b9c88863-2bc6-4f11-9527-a5853720c495'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:56.351061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b9c88863-2bc6-4f11-9527-a5853720c495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab58d5c4c0194b58ad47dd5e4494389f', 'request_id': 'req-b9c88863-2bc6-4f11-9527-a5853720c495'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:56.363264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b9c88863-2bc6-4f11-9527-a5853720c495 admin admin] The token cache is being invalidate because project ab58d5c4c0194b58ad47dd5e4494389f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:00:56.363722 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b9c88863-2bc6-4f11-9527-a5853720c495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:56.364581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b9c88863-2bc6-4f11-9527-a5853720c495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:56.366387 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1244/2476] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:00:56 2022] DELETE /identity/v3/projects/ab58d5c4c0194b58ad47dd5e4494389f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:00:56.371343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05bdb3-faca-4067-8dac-d051a16305b7 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:56.371736 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05bdb3-faca-4067-8dac-d051a16305b7 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:56.372100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05bdb3-faca-4067-8dac-d051a16305b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:56.452795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:00:56.457190 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-df05bdb3-faca-4067-8dac-d051a16305b7 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:00:56.459580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-df05bdb3-faca-4067-8dac-d051a16305b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:56.537699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1233/2477] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:00:56.545770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91793dd-83d9-40ee-babd-0e2a9fd3479a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:56.546080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91793dd-83d9-40ee-babd-0e2a9fd3479a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:56.546341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91793dd-83d9-40ee-babd-0e2a9fd3479a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:56.566086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-d91793dd-83d9-40ee-babd-0e2a9fd3479a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:00:56.567353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d91793dd-83d9-40ee-babd-0e2a9fd3479a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:56.577022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1245/2478] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:00:56.581286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05bdb3-faca-4067-8dac-d051a16305b7 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:56.591737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05bdb3-faca-4067-8dac-d051a16305b7 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:56.596550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dde8f-9401-43a7-9616-63bb1863ad9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:56.597485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dde8f-9401-43a7-9616-63bb1863ad9c admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:56.597485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dde8f-9401-43a7-9616-63bb1863ad9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:56.598392 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906dde8f-9401-43a7-9616-63bb1863ad9c admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:56.599207 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906dde8f-9401-43a7-9616-63bb1863ad9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:00:56.599519 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906dde8f-9401-43a7-9616-63bb1863ad9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:56.601144 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906dde8f-9401-43a7-9616-63bb1863ad9c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:56.613706 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1234/2479] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:00:56.618225 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91793dd-83d9-40ee-babd-0e2a9fd3479a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:56.627628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91793dd-83d9-40ee-babd-0e2a9fd3479a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:56.633038 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe6c85-2ee8-4eec-bb87-4868c506e2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:56.633702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6c85-2ee8-4eec-bb87-4868c506e2cd admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:56.633991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6c85-2ee8-4eec-bb87-4868c506e2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:56.634273 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe6c85-2ee8-4eec-bb87-4868c506e2cd admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:56.634968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe6c85-2ee8-4eec-bb87-4868c506e2cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:56.636918 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe6c85-2ee8-4eec-bb87-4868c506e2cd admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:56.678320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1246/2480] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:00:56 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:00:56.679662 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dde8f-9401-43a7-9616-63bb1863ad9c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:56.691235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906dde8f-9401-43a7-9616-63bb1863ad9c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:56.762828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1cf8a6-6878-40e4-98b3-9f554f5efad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:56.763663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cf8a6-6878-40e4-98b3-9f554f5efad6 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:56.764239 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cf8a6-6878-40e4-98b3-9f554f5efad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:56.764655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1cf8a6-6878-40e4-98b3-9f554f5efad6 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:56.765621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1cf8a6-6878-40e4-98b3-9f554f5efad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:56.768265 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1cf8a6-6878-40e4-98b3-9f554f5efad6 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:56.794094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1235/2481] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:00:56 2022] POST /identity/v3/users => generated 283 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:00:56.798150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe6c85-2ee8-4eec-bb87-4868c506e2cd admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:56.809672 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe6c85-2ee8-4eec-bb87-4868c506e2cd admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:56.814714 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03395c9-8140-4130-a5e3-54ddb85b5d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:56.815398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03395c9-8140-4130-a5e3-54ddb85b5d93 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:56.815687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03395c9-8140-4130-a5e3-54ddb85b5d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:56.816010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03395c9-8140-4130-a5e3-54ddb85b5d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:56.817279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03395c9-8140-4130-a5e3-54ddb85b5d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:56.818890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03395c9-8140-4130-a5e3-54ddb85b5d93 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:56.829220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1247/2482] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:00:56.833972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1cf8a6-6878-40e4-98b3-9f554f5efad6 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:56.844588 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1cf8a6-6878-40e4-98b3-9f554f5efad6 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:56.849660 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ed37b-cc46-46d5-b3b8-f14a1ea98e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:56.850450 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ed37b-cc46-46d5-b3b8-f14a1ea98e63 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:56.850786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ed37b-cc46-46d5-b3b8-f14a1ea98e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:56.851134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ed37b-cc46-46d5-b3b8-f14a1ea98e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:56.851977 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ed37b-cc46-46d5-b3b8-f14a1ea98e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:56.853651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ed37b-cc46-46d5-b3b8-f14a1ea98e63 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:56.864347 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1236/2483] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:00:56.868461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03395c9-8140-4130-a5e3-54ddb85b5d93 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:56.878339 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03395c9-8140-4130-a5e3-54ddb85b5d93 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:56.883522 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf07b94-4fd3-43b6-a7c8-f970482e727e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:56.884474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf07b94-4fd3-43b6-a7c8-f970482e727e admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:56.884773 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf07b94-4fd3-43b6-a7c8-f970482e727e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:56.885050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf07b94-4fd3-43b6-a7c8-f970482e727e admin admin] PATH_INFO: `/v3/projects/e6d31221a5b1457ca00c6be809936331/users/d2a4f0201f1240758ef554054a260914/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:56.901036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf07b94-4fd3-43b6-a7c8-f970482e727e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d31221a5b1457ca00c6be809936331, user_id=d2a4f0201f1240758ef554054a260914, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:56.903660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf07b94-4fd3-43b6-a7c8-f970482e727e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:56.917664 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1248/2484] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:00:56 2022] PUT /identity/v3/projects/e6d31221a5b1457ca00c6be809936331/users/d2a4f0201f1240758ef554054a260914/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:00:56.927592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ce6d0-d92f-4622-98b1-a4ce16b726a1 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:56.927884 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ce6d0-d92f-4622-98b1-a4ce16b726a1 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:56.928208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ce6d0-d92f-4622-98b1-a4ce16b726a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:56.954537 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-671ce6d0-d92f-4622-98b1-a4ce16b726a1 None None] MFA Rules not processed for user `d2a4f0201f1240758ef554054a260914`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:00:56.955738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-671ce6d0-d92f-4622-98b1-a4ce16b726a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:57.020795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1237/2485] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:00:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:00:57.025367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf07b94-4fd3-43b6-a7c8-f970482e727e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:57.030908 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:00:57.089478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf07b94-4fd3-43b6-a7c8-f970482e727e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:57.094171 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b579a-373f-4180-aaa1-7273288d319d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:57.094844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b579a-373f-4180-aaa1-7273288d319d admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:57.095130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b579a-373f-4180-aaa1-7273288d319d admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:57.095411 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b579a-373f-4180-aaa1-7273288d319d admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:57.096127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b579a-373f-4180-aaa1-7273288d319d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:57.098178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b579a-373f-4180-aaa1-7273288d319d admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:57.125337 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1249/2486] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:00:57 2022] POST /identity/v3/projects => generated 363 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:00:57.129975 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ce6d0-d92f-4622-98b1-a4ce16b726a1 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:57.135294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:00:57.142517 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ce6d0-d92f-4622-98b1-a4ce16b726a1 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:57.199671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dfb96-bf0f-49e1-987f-72abbe02a836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:57.200382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dfb96-bf0f-49e1-987f-72abbe02a836 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:57.200670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dfb96-bf0f-49e1-987f-72abbe02a836 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:57.200950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27dfb96-bf0f-49e1-987f-72abbe02a836 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:57.201649 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dfb96-bf0f-49e1-987f-72abbe02a836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:57.203564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27dfb96-bf0f-49e1-987f-72abbe02a836 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:57.222981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1238/2487] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:00:57 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:00:57.226955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b579a-373f-4180-aaa1-7273288d319d admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:57.237213 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b579a-373f-4180-aaa1-7273288d319d admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:57.242247 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54284efe-aa5b-4b5c-bc6f-c49465264c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:57.242900 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54284efe-aa5b-4b5c-bc6f-c49465264c2d admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:57.243185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54284efe-aa5b-4b5c-bc6f-c49465264c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:57.243461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54284efe-aa5b-4b5c-bc6f-c49465264c2d admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:57.244208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54284efe-aa5b-4b5c-bc6f-c49465264c2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:57.245826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54284efe-aa5b-4b5c-bc6f-c49465264c2d admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:57.257281 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1250/2488] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:00:57.261752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dfb96-bf0f-49e1-987f-72abbe02a836 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:57.272379 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27dfb96-bf0f-49e1-987f-72abbe02a836 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:57.277043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac0a47-4e93-4cc1-9278-0777e9d2875c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:57.277932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac0a47-4e93-4cc1-9278-0777e9d2875c admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:57.278210 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac0a47-4e93-4cc1-9278-0777e9d2875c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:57.278478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac0a47-4e93-4cc1-9278-0777e9d2875c admin admin] PATH_INFO: `/v3/projects/d4569a85e247477e98d911fcc1e5a81c/users/3cc483a37f8045a0accebef8a2bd7819/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:57.293067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac0a47-4e93-4cc1-9278-0777e9d2875c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4569a85e247477e98d911fcc1e5a81c, user_id=3cc483a37f8045a0accebef8a2bd7819, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:57.296024 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ac0a47-4e93-4cc1-9278-0777e9d2875c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:57.309166 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1239/2489] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:00:57 2022] PUT /identity/v3/projects/d4569a85e247477e98d911fcc1e5a81c/users/3cc483a37f8045a0accebef8a2bd7819/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:00:57.313310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54284efe-aa5b-4b5c-bc6f-c49465264c2d admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:57.324028 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54284efe-aa5b-4b5c-bc6f-c49465264c2d admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:57.381572 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff924d-d53b-4329-94d3-3f5495fe8099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:57.382255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff924d-d53b-4329-94d3-3f5495fe8099 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:57.382539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff924d-d53b-4329-94d3-3f5495fe8099 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:57.382819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff924d-d53b-4329-94d3-3f5495fe8099 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:57.383550 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff924d-d53b-4329-94d3-3f5495fe8099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:57.385175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff924d-d53b-4329-94d3-3f5495fe8099 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:57.394834 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1251/2490] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:00:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 18:00:57.399623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac0a47-4e93-4cc1-9278-0777e9d2875c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:57.409423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac0a47-4e93-4cc1-9278-0777e9d2875c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:57.414651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38bb6c-6ff0-4533-8bc3-6242fc6d3d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:57.415528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38bb6c-6ff0-4533-8bc3-6242fc6d3d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:57.415807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38bb6c-6ff0-4533-8bc3-6242fc6d3d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:57.416113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c38bb6c-6ff0-4533-8bc3-6242fc6d3d18 admin admin] PATH_INFO: `/v3/projects/d4569a85e247477e98d911fcc1e5a81c/users/3cc483a37f8045a0accebef8a2bd7819/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:57.418152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38bb6c-6ff0-4533-8bc3-6242fc6d3d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4569a85e247477e98d911fcc1e5a81c, user_id=3cc483a37f8045a0accebef8a2bd7819, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:57.420940 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c38bb6c-6ff0-4533-8bc3-6242fc6d3d18 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:57.432957 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1240/2491] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:00:57 2022] PUT /identity/v3/projects/d4569a85e247477e98d911fcc1e5a81c/users/3cc483a37f8045a0accebef8a2bd7819/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:00:57.442690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33586fb-50d0-423e-aac1-7d206c06f430 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:57.442983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33586fb-50d0-423e-aac1-7d206c06f430 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:57.443265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33586fb-50d0-423e-aac1-7d206c06f430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:57.469858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f33586fb-50d0-423e-aac1-7d206c06f430 None None] MFA Rules not processed for user `3cc483a37f8045a0accebef8a2bd7819`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:00:57.471061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f33586fb-50d0-423e-aac1-7d206c06f430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:57.535803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:00:57.540163 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1252/2492] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:00:57 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:00:57.553329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38bb6c-6ff0-4533-8bc3-6242fc6d3d18 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:57.555048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1c38bb6c-6ff0-4533-8bc3-6242fc6d3d18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:57.632873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c38bb6c-6ff0-4533-8bc3-6242fc6d3d18 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:57.637226 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df7397-2bca-4f54-ba6a-de2ba29b3f9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:57.637874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df7397-2bca-4f54-ba6a-de2ba29b3f9d service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:57.638154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df7397-2bca-4f54-ba6a-de2ba29b3f9d service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:57.638428 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df7397-2bca-4f54-ba6a-de2ba29b3f9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:57.647602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df7397-2bca-4f54-ba6a-de2ba29b3f9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:57.649053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df7397-2bca-4f54-ba6a-de2ba29b3f9d service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:57.658461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1241/2493] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:00:57 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:00:58.134793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33586fb-50d0-423e-aac1-7d206c06f430 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:58.208793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33586fb-50d0-423e-aac1-7d206c06f430 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:58.216841 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60da055-f170-4d24-a218-d36249554c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:58.217697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60da055-f170-4d24-a218-d36249554c92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:58.217989 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60da055-f170-4d24-a218-d36249554c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:58.218267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60da055-f170-4d24-a218-d36249554c92 admin admin] PATH_INFO: `/v3/users/d2a4f0201f1240758ef554054a260914` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:58.220280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60da055-f170-4d24-a218-d36249554c92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2a4f0201f1240758ef554054a260914)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:58.225655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60da055-f170-4d24-a218-d36249554c92 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:58.280182 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-a60da055-f170-4d24-a218-d36249554c92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2a4f0201f1240758ef554054a260914'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:58.292290 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-a60da055-f170-4d24-a218-d36249554c92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2a4f0201f1240758ef554054a260914'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:58.301401 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-a60da055-f170-4d24-a218-d36249554c92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2a4f0201f1240758ef554054a260914'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:58.302038 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-a60da055-f170-4d24-a218-d36249554c92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2a4f0201f1240758ef554054a260914'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:58.309864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-a60da055-f170-4d24-a218-d36249554c92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2a4f0201f1240758ef554054a260914'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:58.316312 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1253/2494] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:00:58 2022] DELETE /identity/v3/users/d2a4f0201f1240758ef554054a260914 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:00:58.320936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b50a6c-0fbe-4679-8278-afa3e2f3ebea None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:58.321411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b50a6c-0fbe-4679-8278-afa3e2f3ebea None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:58.321529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b50a6c-0fbe-4679-8278-afa3e2f3ebea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:58.340505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-b0b50a6c-0fbe-4679-8278-afa3e2f3ebea None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:00:58.341749 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b0b50a6c-0fbe-4679-8278-afa3e2f3ebea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:58.374609 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:00:58.403248 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1242/2495] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:00:58.412906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60da055-f170-4d24-a218-d36249554c92 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:58.414359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a60da055-f170-4d24-a218-d36249554c92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:58.472971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60da055-f170-4d24-a218-d36249554c92 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:58.476674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79db34-11cc-4721-a82c-b73e5688e565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:58.479690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79db34-11cc-4721-a82c-b73e5688e565 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:58.483797 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79db34-11cc-4721-a82c-b73e5688e565 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:58.484582 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e79db34-11cc-4721-a82c-b73e5688e565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:58.495651 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79db34-11cc-4721-a82c-b73e5688e565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:58.497127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e79db34-11cc-4721-a82c-b73e5688e565 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:58.507259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1254/2496] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:00:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:00:58.838376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b50a6c-0fbe-4679-8278-afa3e2f3ebea None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:58.839966 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b0b50a6c-0fbe-4679-8278-afa3e2f3ebea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:58.857066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b50a6c-0fbe-4679-8278-afa3e2f3ebea None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:58.863354 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:58.864388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:58.864764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:58.865142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] PATH_INFO: `/v3/projects/e6d31221a5b1457ca00c6be809936331` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:58.866461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d31221a5b1457ca00c6be809936331)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:58.868587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:58.943694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d31221a5b1457ca00c6be809936331', 'request_id': 'req-b14312ca-9857-40d7-b830-dd6b4244fbde'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:58.958193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d31221a5b1457ca00c6be809936331', 'request_id': 'req-b14312ca-9857-40d7-b830-dd6b4244fbde'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:58.968579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d31221a5b1457ca00c6be809936331', 'request_id': 'req-b14312ca-9857-40d7-b830-dd6b4244fbde'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:58.985875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] The token cache is being invalidate because project e6d31221a5b1457ca00c6be809936331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:00:58.987113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:58.992176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:58.994384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1243/2497] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:00:58 2022] DELETE /identity/v3/projects/e6d31221a5b1457ca00c6be809936331 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:00:59.002017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79db34-11cc-4721-a82c-b73e5688e565 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:59.023530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e79db34-11cc-4721-a82c-b73e5688e565 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:59.052513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:00:59.103653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d0b37-7de6-40ce-8184-f4c0b7f7c428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:59.104595 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d0b37-7de6-40ce-8184-f4c0b7f7c428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:59.104882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d0b37-7de6-40ce-8184-f4c0b7f7c428 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:59.105163 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d0b37-7de6-40ce-8184-f4c0b7f7c428 admin admin] PATH_INFO: `/v3/users/3cc483a37f8045a0accebef8a2bd7819` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:59.106365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d0b37-7de6-40ce-8184-f4c0b7f7c428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc483a37f8045a0accebef8a2bd7819)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:59.108553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47d0b37-7de6-40ce-8184-f4c0b7f7c428 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:59.171369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-c47d0b37-7de6-40ce-8184-f4c0b7f7c428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc483a37f8045a0accebef8a2bd7819'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:59.185979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-c47d0b37-7de6-40ce-8184-f4c0b7f7c428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc483a37f8045a0accebef8a2bd7819'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:59.197729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-c47d0b37-7de6-40ce-8184-f4c0b7f7c428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc483a37f8045a0accebef8a2bd7819'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:59.198717 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-c47d0b37-7de6-40ce-8184-f4c0b7f7c428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc483a37f8045a0accebef8a2bd7819'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:59.209147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-c47d0b37-7de6-40ce-8184-f4c0b7f7c428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc483a37f8045a0accebef8a2bd7819'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:59.221303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1255/2498] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:00:59 2022] DELETE /identity/v3/users/3cc483a37f8045a0accebef8a2bd7819 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:00:59.582028 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:59.583442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:59.604597 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14312ca-9857-40d7-b830-dd6b4244fbde admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:59.667238 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd0e22-444d-41f0-8f84-f46f7276f5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:59.668297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd0e22-444d-41f0-8f84-f46f7276f5a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:59.668635 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd0e22-444d-41f0-8f84-f46f7276f5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:59.668924 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd0e22-444d-41f0-8f84-f46f7276f5a2 admin admin] PATH_INFO: `/v3/projects/d4569a85e247477e98d911fcc1e5a81c` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:59.670571 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd0e22-444d-41f0-8f84-f46f7276f5a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4569a85e247477e98d911fcc1e5a81c)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:59.674097 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd0e22-444d-41f0-8f84-f46f7276f5a2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:59.735078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-97bd0e22-444d-41f0-8f84-f46f7276f5a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4569a85e247477e98d911fcc1e5a81c', 'request_id': 'req-97bd0e22-444d-41f0-8f84-f46f7276f5a2'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:59.744475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-97bd0e22-444d-41f0-8f84-f46f7276f5a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4569a85e247477e98d911fcc1e5a81c', 'request_id': 'req-97bd0e22-444d-41f0-8f84-f46f7276f5a2'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:59.750174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:00:59.752520 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-97bd0e22-444d-41f0-8f84-f46f7276f5a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4569a85e247477e98d911fcc1e5a81c', 'request_id': 'req-97bd0e22-444d-41f0-8f84-f46f7276f5a2'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:59.768981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-97bd0e22-444d-41f0-8f84-f46f7276f5a2 admin admin] The token cache is being invalidate because project d4569a85e247477e98d911fcc1e5a81c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:00:59.769282 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-97bd0e22-444d-41f0-8f84-f46f7276f5a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:59.769932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-97bd0e22-444d-41f0-8f84-f46f7276f5a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:00:59.771549 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1244/2499] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:00:59 2022] DELETE /identity/v3/projects/d4569a85e247477e98d911fcc1e5a81c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:00:59.779445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135365ed-fdc5-4f17-b99f-fa60734743d5 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:59.779987 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135365ed-fdc5-4f17-b99f-fa60734743d5 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:59.780164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135365ed-fdc5-4f17-b99f-fa60734743d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:59.798748 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-135365ed-fdc5-4f17-b99f-fa60734743d5 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:00:59.800105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-135365ed-fdc5-4f17-b99f-fa60734743d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:59.866267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1256/2500] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:00:59.874134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5069-7bd7-4e8a-8c4f-7ddf4cc5f71d None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:59.874453 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5069-7bd7-4e8a-8c4f-7ddf4cc5f71d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:59.874734 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c5069-7bd7-4e8a-8c4f-7ddf4cc5f71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:59.894528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-491c5069-7bd7-4e8a-8c4f-7ddf4cc5f71d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:00:59.895725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-491c5069-7bd7-4e8a-8c4f-7ddf4cc5f71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:00:59.903830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1245/2501] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:00:59.908301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135365ed-fdc5-4f17-b99f-fa60734743d5 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:59.918153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135365ed-fdc5-4f17-b99f-fa60734743d5 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:59.922720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ad92a-26a9-44cc-a065-2e29ba2529cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:59.923408 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ad92a-26a9-44cc-a065-2e29ba2529cb admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:59.923692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ad92a-26a9-44cc-a065-2e29ba2529cb admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:59.924001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ad92a-26a9-44cc-a065-2e29ba2529cb admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:59.924753 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ad92a-26a9-44cc-a065-2e29ba2529cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:00:59.925059 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ad92a-26a9-44cc-a065-2e29ba2529cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:59.926636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ad92a-26a9-44cc-a065-2e29ba2529cb admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:59.939705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1257/2502] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:00:59.944196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c5069-7bd7-4e8a-8c4f-7ddf4cc5f71d None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:00:59.953516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c5069-7bd7-4e8a-8c4f-7ddf4cc5f71d None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:00:59.958790 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deec3b6-0db9-403d-954b-2a74ec98a08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:00:59.959449 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deec3b6-0db9-403d-954b-2a74ec98a08a admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:00:59.959737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deec3b6-0db9-403d-954b-2a74ec98a08a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:00:59.960057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deec3b6-0db9-403d-954b-2a74ec98a08a admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:00:59.960755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deec3b6-0db9-403d-954b-2a74ec98a08a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:00:59.962664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deec3b6-0db9-403d-954b-2a74ec98a08a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:00:59.991156 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1246/2503] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:00:59 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:00:59.992361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ad92a-26a9-44cc-a065-2e29ba2529cb admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:00.003149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ad92a-26a9-44cc-a065-2e29ba2529cb admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:00.029680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:00.075932 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66051d5-6ad1-408c-93bd-019bea6fd46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:00.076668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66051d5-6ad1-408c-93bd-019bea6fd46b admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:00.076957 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66051d5-6ad1-408c-93bd-019bea6fd46b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:00.077236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66051d5-6ad1-408c-93bd-019bea6fd46b admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:00.078085 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66051d5-6ad1-408c-93bd-019bea6fd46b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:00.080071 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66051d5-6ad1-408c-93bd-019bea6fd46b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:00.106220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1258/2504] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:00:59 2022] POST /identity/v3/users => generated 285 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:01:00.107399 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deec3b6-0db9-403d-954b-2a74ec98a08a admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:00.119350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deec3b6-0db9-403d-954b-2a74ec98a08a admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:00.124202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3c689-ab68-4d01-9a16-4de391b59439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:00.124888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3c689-ab68-4d01-9a16-4de391b59439 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:00.125178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3c689-ab68-4d01-9a16-4de391b59439 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:00.125575 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad3c689-ab68-4d01-9a16-4de391b59439 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:00.126818 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3c689-ab68-4d01-9a16-4de391b59439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:00.128473 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad3c689-ab68-4d01-9a16-4de391b59439 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:00.139480 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1247/2505] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:01:00.143690 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66051d5-6ad1-408c-93bd-019bea6fd46b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:00.154150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66051d5-6ad1-408c-93bd-019bea6fd46b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:00.161339 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35168add-b6a0-409b-8281-343b54b8b732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:00.162021 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35168add-b6a0-409b-8281-343b54b8b732 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:00.162306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35168add-b6a0-409b-8281-343b54b8b732 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:00.162586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35168add-b6a0-409b-8281-343b54b8b732 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:00.163448 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35168add-b6a0-409b-8281-343b54b8b732 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:00.165105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35168add-b6a0-409b-8281-343b54b8b732 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:00.178175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1259/2506] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:01:00.183071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3c689-ab68-4d01-9a16-4de391b59439 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:00.194125 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad3c689-ab68-4d01-9a16-4de391b59439 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:00.205356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b7f91-090a-4f11-8424-2d32c42285b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:00.206514 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b7f91-090a-4f11-8424-2d32c42285b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:00.206803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b7f91-090a-4f11-8424-2d32c42285b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:00.207084 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b7f91-090a-4f11-8424-2d32c42285b9 admin admin] PATH_INFO: `/v3/projects/34da770e07654d55b99c33ae55a4765f/users/d3fd7490742e49d6ab753eca99d22399/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:00.224743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b7f91-090a-4f11-8424-2d32c42285b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34da770e07654d55b99c33ae55a4765f, user_id=d3fd7490742e49d6ab753eca99d22399, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:00.227716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b7f91-090a-4f11-8424-2d32c42285b9 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:00.247353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1248/2507] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:01:00 2022] PUT /identity/v3/projects/34da770e07654d55b99c33ae55a4765f/users/d3fd7490742e49d6ab753eca99d22399/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:00.258433 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51184893-9527-4bf8-b627-081579111b44 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:00.258727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51184893-9527-4bf8-b627-081579111b44 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:00.259012 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51184893-9527-4bf8-b627-081579111b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:00.286340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-51184893-9527-4bf8-b627-081579111b44 None None] MFA Rules not processed for user `d3fd7490742e49d6ab753eca99d22399`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:00.287949 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-51184893-9527-4bf8-b627-081579111b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:00.353412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1260/2508] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:01:00.357985 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b7f91-090a-4f11-8424-2d32c42285b9 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:00.421954 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b7f91-090a-4f11-8424-2d32c42285b9 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:00.427461 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc3202-058e-45be-aaa7-eb48e4e6eb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:00.428155 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc3202-058e-45be-aaa7-eb48e4e6eb91 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:00.428448 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc3202-058e-45be-aaa7-eb48e4e6eb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:00.428723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadc3202-058e-45be-aaa7-eb48e4e6eb91 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:00.429685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc3202-058e-45be-aaa7-eb48e4e6eb91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:00.431596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadc3202-058e-45be-aaa7-eb48e4e6eb91 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:00.457904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1249/2509] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:01:00 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:01:00.461992 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51184893-9527-4bf8-b627-081579111b44 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:00.473704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51184893-9527-4bf8-b627-081579111b44 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:00.540991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b090a8b-b87a-4252-97d7-163bd6f38c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:00.540991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b090a8b-b87a-4252-97d7-163bd6f38c8d admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:00.540991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b090a8b-b87a-4252-97d7-163bd6f38c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:00.540991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b090a8b-b87a-4252-97d7-163bd6f38c8d admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:00.541408 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b090a8b-b87a-4252-97d7-163bd6f38c8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:00.543227 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b090a8b-b87a-4252-97d7-163bd6f38c8d admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:00.566437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1261/2510] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:01:00 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:01:00.567703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc3202-058e-45be-aaa7-eb48e4e6eb91 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:00.578183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadc3202-058e-45be-aaa7-eb48e4e6eb91 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:00.583568 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7ab6e-2ca9-4e52-a8de-43791777a1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:00.584272 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7ab6e-2ca9-4e52-a8de-43791777a1be admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:00.584589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7ab6e-2ca9-4e52-a8de-43791777a1be admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:00.584871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7ab6e-2ca9-4e52-a8de-43791777a1be admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:00.585605 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7ab6e-2ca9-4e52-a8de-43791777a1be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:00.587219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b7ab6e-2ca9-4e52-a8de-43791777a1be admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:00.598641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1250/2511] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:01:00.603183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b090a8b-b87a-4252-97d7-163bd6f38c8d admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:00.614055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b090a8b-b87a-4252-97d7-163bd6f38c8d admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:00.618814 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4485d1-8c60-446b-b886-c44cd0b77bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:00.619694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4485d1-8c60-446b-b886-c44cd0b77bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:00.620014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4485d1-8c60-446b-b886-c44cd0b77bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:00.620296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4485d1-8c60-446b-b886-c44cd0b77bd3 admin admin] PATH_INFO: `/v3/projects/43c77c76601e4dd49c34d7544cb63b54/users/26c88e5bb7af4be28fdc22ae1a915728/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:00.634878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4485d1-8c60-446b-b886-c44cd0b77bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c77c76601e4dd49c34d7544cb63b54, user_id=26c88e5bb7af4be28fdc22ae1a915728, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:00.637520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4485d1-8c60-446b-b886-c44cd0b77bd3 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:00.649967 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1262/2512] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:01:00 2022] PUT /identity/v3/projects/43c77c76601e4dd49c34d7544cb63b54/users/26c88e5bb7af4be28fdc22ae1a915728/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:00.653981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7ab6e-2ca9-4e52-a8de-43791777a1be admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:00.664782 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7ab6e-2ca9-4e52-a8de-43791777a1be admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:00.722852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3123df8d-afd9-48ec-85ee-9f36939aa74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:00.723535 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123df8d-afd9-48ec-85ee-9f36939aa74d admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:00.723845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123df8d-afd9-48ec-85ee-9f36939aa74d admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:00.724133 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3123df8d-afd9-48ec-85ee-9f36939aa74d admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:00.724885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123df8d-afd9-48ec-85ee-9f36939aa74d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:00.726781 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3123df8d-afd9-48ec-85ee-9f36939aa74d admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:00.729111 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:00.738525 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1251/2513] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:01:00 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:01:00.744558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4485d1-8c60-446b-b886-c44cd0b77bd3 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:00.750700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:00.756833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4485d1-8c60-446b-b886-c44cd0b77bd3 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:00.763017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176bcc4-bc5b-454c-90e6-0c8f8584283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:00.764214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176bcc4-bc5b-454c-90e6-0c8f8584283a admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:00.764604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176bcc4-bc5b-454c-90e6-0c8f8584283a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:00.764914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176bcc4-bc5b-454c-90e6-0c8f8584283a admin admin] PATH_INFO: `/v3/projects/43c77c76601e4dd49c34d7544cb63b54/users/26c88e5bb7af4be28fdc22ae1a915728/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:00.768209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176bcc4-bc5b-454c-90e6-0c8f8584283a admin admin] RBAC: Authorizing `identity:create_grant(project_id=43c77c76601e4dd49c34d7544cb63b54, user_id=26c88e5bb7af4be28fdc22ae1a915728, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:00.770868 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0176bcc4-bc5b-454c-90e6-0c8f8584283a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:00.782699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1263/2514] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:01:00 2022] PUT /identity/v3/projects/43c77c76601e4dd49c34d7544cb63b54/users/26c88e5bb7af4be28fdc22ae1a915728/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:00.793844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45cb40-7b6a-4faa-ae1a-9fc8e592b086 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:00.794160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45cb40-7b6a-4faa-ae1a-9fc8e592b086 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:00.794438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45cb40-7b6a-4faa-ae1a-9fc8e592b086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:00.821334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-0f45cb40-7b6a-4faa-ae1a-9fc8e592b086 None None] MFA Rules not processed for user `26c88e5bb7af4be28fdc22ae1a915728`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:00.822572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0f45cb40-7b6a-4faa-ae1a-9fc8e592b086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:00.890552 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1252/2515] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:00 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:01:00.901836 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176bcc4-bc5b-454c-90e6-0c8f8584283a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:00.903830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0176bcc4-bc5b-454c-90e6-0c8f8584283a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:00.961537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176bcc4-bc5b-454c-90e6-0c8f8584283a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:00.965211 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d8caa-90ca-4f22-97a6-6663b29dadcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:00.965865 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d8caa-90ca-4f22-97a6-6663b29dadcd service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:00.966148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d8caa-90ca-4f22-97a6-6663b29dadcd service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:00.966431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d8caa-90ca-4f22-97a6-6663b29dadcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:00.976266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d8caa-90ca-4f22-97a6-6663b29dadcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:00.977698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329d8caa-90ca-4f22-97a6-6663b29dadcd service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:00.987056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1264/2516] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:01:00 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:01:02.228447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45cb40-7b6a-4faa-ae1a-9fc8e592b086 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:02.229894 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0f45cb40-7b6a-4faa-ae1a-9fc8e592b086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:02.341640 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45cb40-7b6a-4faa-ae1a-9fc8e592b086 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:02.348383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc2004-a95b-4679-ad79-c04ae3910dbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:02.349056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc2004-a95b-4679-ad79-c04ae3910dbd service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:02.349344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc2004-a95b-4679-ad79-c04ae3910dbd service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:02.349619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc2004-a95b-4679-ad79-c04ae3910dbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:02.351537 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d2dc2004-a95b-4679-ad79-c04ae3910dbd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:02.422867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc2004-a95b-4679-ad79-c04ae3910dbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:02.426389 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc2004-a95b-4679-ad79-c04ae3910dbd service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:02.435565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d8caa-90ca-4f22-97a6-6663b29dadcd service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:02.443544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1253/2517] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:01:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:01:02.545485 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d8caa-90ca-4f22-97a6-6663b29dadcd service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:02.550730 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fc537-7d49-4b68-a224-c68b16b84e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:02.551656 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fc537-7d49-4b68-a224-c68b16b84e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:02.552219 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fc537-7d49-4b68-a224-c68b16b84e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:02.552555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546fc537-7d49-4b68-a224-c68b16b84e39 admin admin] PATH_INFO: `/v3/users/d3fd7490742e49d6ab753eca99d22399` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:02.554013 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546fc537-7d49-4b68-a224-c68b16b84e39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3fd7490742e49d6ab753eca99d22399)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:02.556486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546fc537-7d49-4b68-a224-c68b16b84e39 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:02.558344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7380d-5cc1-4b53-b53a-bf8d8d0c7486 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:02.558637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7380d-5cc1-4b53-b53a-bf8d8d0c7486 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:02.558922 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7380d-5cc1-4b53-b53a-bf8d8d0c7486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:02.579172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-54e7380d-5cc1-4b53-b53a-bf8d8d0c7486 None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:02.580456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-54e7380d-5cc1-4b53-b53a-bf8d8d0c7486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:02.585420 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:02.624375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-546fc537-7d49-4b68-a224-c68b16b84e39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3fd7490742e49d6ab753eca99d22399'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:02.647760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-546fc537-7d49-4b68-a224-c68b16b84e39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3fd7490742e49d6ab753eca99d22399'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:02.649384 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1254/2518] 10.222.0.64 () {58 vars in 967 bytes} [Thu Jul 14 18:01:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:01:02.659090 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-546fc537-7d49-4b68-a224-c68b16b84e39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3fd7490742e49d6ab753eca99d22399'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:02.659728 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-546fc537-7d49-4b68-a224-c68b16b84e39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3fd7490742e49d6ab753eca99d22399'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:02.669313 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-546fc537-7d49-4b68-a224-c68b16b84e39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3fd7490742e49d6ab753eca99d22399'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:02.677353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1265/2519] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:01:02 2022] DELETE /identity/v3/users/d3fd7490742e49d6ab753eca99d22399 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:02.681825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c83f62-a12a-4d94-bc64-2fbf16b38dd7 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:02.682058 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c83f62-a12a-4d94-bc64-2fbf16b38dd7 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:02.682344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c83f62-a12a-4d94-bc64-2fbf16b38dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:02.705691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-62c83f62-a12a-4d94-bc64-2fbf16b38dd7 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:02.707918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-62c83f62-a12a-4d94-bc64-2fbf16b38dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:02.711035 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fc537-7d49-4b68-a224-c68b16b84e39 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:02.712476 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-546fc537-7d49-4b68-a224-c68b16b84e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:02.729000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:02.781844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1255/2520] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:01:02.789723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c83f62-a12a-4d94-bc64-2fbf16b38dd7 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:02.791180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-62c83f62-a12a-4d94-bc64-2fbf16b38dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:02.799571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546fc537-7d49-4b68-a224-c68b16b84e39 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:02.805209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f6649-5217-4610-b453-8d236179fb91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:02.806004 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f6649-5217-4610-b453-8d236179fb91 service swift] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:02.806424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f6649-5217-4610-b453-8d236179fb91 service swift] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:02.806611 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636f6649-5217-4610-b453-8d236179fb91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:02.808602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-636f6649-5217-4610-b453-8d236179fb91 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:02.864056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c83f62-a12a-4d94-bc64-2fbf16b38dd7 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:02.867960 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad84b80-225c-41de-a252-74ef5d1d979b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:02.868656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad84b80-225c-41de-a252-74ef5d1d979b service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:02.868946 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad84b80-225c-41de-a252-74ef5d1d979b service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:02.869231 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad84b80-225c-41de-a252-74ef5d1d979b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:02.884138 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad84b80-225c-41de-a252-74ef5d1d979b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:02.886305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad84b80-225c-41de-a252-74ef5d1d979b service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:02.898167 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1256/2521] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:01:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:01:02.905239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f6649-5217-4610-b453-8d236179fb91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:02.907732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636f6649-5217-4610-b453-8d236179fb91 service swift] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:02.920207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1266/2522] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 18:01:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 18:01:03.237938 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad84b80-225c-41de-a252-74ef5d1d979b service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:03.240643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-bad84b80-225c-41de-a252-74ef5d1d979b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:03.264899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad84b80-225c-41de-a252-74ef5d1d979b service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:03.270336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:03.271412 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:03.271637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:03.271943 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] PATH_INFO: `/v3/projects/34da770e07654d55b99c33ae55a4765f` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:03.273723 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34da770e07654d55b99c33ae55a4765f)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:03.275942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:03.355343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34da770e07654d55b99c33ae55a4765f', 'request_id': 'req-3e4e6be5-a47e-41ce-bd04-60337122e2d3'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:03.358596 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:03.366750 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34da770e07654d55b99c33ae55a4765f', 'request_id': 'req-3e4e6be5-a47e-41ce-bd04-60337122e2d3'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:03.377232 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34da770e07654d55b99c33ae55a4765f', 'request_id': 'req-3e4e6be5-a47e-41ce-bd04-60337122e2d3'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:03.383398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f6649-5217-4610-b453-8d236179fb91 service swift] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:03.394431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] The token cache is being invalidate because project 34da770e07654d55b99c33ae55a4765f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:01:03.395196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:03.395539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:03.396036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636f6649-5217-4610-b453-8d236179fb91 service swift] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:03.397724 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1257/2523] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:01:03 2022] DELETE /identity/v3/projects/34da770e07654d55b99c33ae55a4765f => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:03.401509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:03.402970 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:03.459524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be641a93-355b-4cab-b890-d8ad80e7cf3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:03.460326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be641a93-355b-4cab-b890-d8ad80e7cf3f service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:03.460661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be641a93-355b-4cab-b890-d8ad80e7cf3f service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:03.461009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be641a93-355b-4cab-b890-d8ad80e7cf3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:03.463889 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-be641a93-355b-4cab-b890-d8ad80e7cf3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:03.490012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e6be5-a47e-41ce-bd04-60337122e2d3 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:03.496080 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9a2c4-5141-4685-acd9-b9cc6e91139c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:03.497066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9a2c4-5141-4685-acd9-b9cc6e91139c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:03.497544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9a2c4-5141-4685-acd9-b9cc6e91139c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:03.498481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9a2c4-5141-4685-acd9-b9cc6e91139c admin admin] PATH_INFO: `/v3/users/26c88e5bb7af4be28fdc22ae1a915728` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:03.499834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9a2c4-5141-4685-acd9-b9cc6e91139c admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c88e5bb7af4be28fdc22ae1a915728)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:03.501871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9a2c4-5141-4685-acd9-b9cc6e91139c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:03.556261 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be641a93-355b-4cab-b890-d8ad80e7cf3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:03.557856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be641a93-355b-4cab-b890-d8ad80e7cf3f service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:03.568786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-dcc9a2c4-5141-4685-acd9-b9cc6e91139c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c88e5bb7af4be28fdc22ae1a915728'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:03.579576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1267/2524] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:01:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 18:01:03.594823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-dcc9a2c4-5141-4685-acd9-b9cc6e91139c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c88e5bb7af4be28fdc22ae1a915728'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:03.595905 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-dcc9a2c4-5141-4685-acd9-b9cc6e91139c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c88e5bb7af4be28fdc22ae1a915728'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:03.609932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-dcc9a2c4-5141-4685-acd9-b9cc6e91139c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c88e5bb7af4be28fdc22ae1a915728'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:03.610758 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-dcc9a2c4-5141-4685-acd9-b9cc6e91139c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c88e5bb7af4be28fdc22ae1a915728'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:03.628646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1258/2525] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:01:03 2022] DELETE /identity/v3/users/26c88e5bb7af4be28fdc22ae1a915728 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:03.984164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be641a93-355b-4cab-b890-d8ad80e7cf3f service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:03.996618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=100 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:04.004297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be641a93-355b-4cab-b890-d8ad80e7cf3f service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:04.090361 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31897a6-14eb-495a-9d83-4226f0a54811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:04.092148 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31897a6-14eb-495a-9d83-4226f0a54811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:04.092616 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31897a6-14eb-495a-9d83-4226f0a54811 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:04.093231 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31897a6-14eb-495a-9d83-4226f0a54811 admin admin] PATH_INFO: `/v3/projects/43c77c76601e4dd49c34d7544cb63b54` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:04.097928 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31897a6-14eb-495a-9d83-4226f0a54811 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43c77c76601e4dd49c34d7544cb63b54)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:04.100254 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31897a6-14eb-495a-9d83-4226f0a54811 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:04.163178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f31897a6-14eb-495a-9d83-4226f0a54811 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43c77c76601e4dd49c34d7544cb63b54', 'request_id': 'req-f31897a6-14eb-495a-9d83-4226f0a54811'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:04.175858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f31897a6-14eb-495a-9d83-4226f0a54811 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43c77c76601e4dd49c34d7544cb63b54', 'request_id': 'req-f31897a6-14eb-495a-9d83-4226f0a54811'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:04.183909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f31897a6-14eb-495a-9d83-4226f0a54811 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43c77c76601e4dd49c34d7544cb63b54', 'request_id': 'req-f31897a6-14eb-495a-9d83-4226f0a54811'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:04.195664 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f31897a6-14eb-495a-9d83-4226f0a54811 admin admin] The token cache is being invalidate because project 43c77c76601e4dd49c34d7544cb63b54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:01:04.196059 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f31897a6-14eb-495a-9d83-4226f0a54811 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:04.196698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f31897a6-14eb-495a-9d83-4226f0a54811 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:04.198509 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1268/2526] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:01:03 2022] DELETE /identity/v3/projects/43c77c76601e4dd49c34d7544cb63b54 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:01:04.203736 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83560b1-7501-4dba-ab69-f7ea63afe647 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:04.204171 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83560b1-7501-4dba-ab69-f7ea63afe647 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:04.204557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83560b1-7501-4dba-ab69-f7ea63afe647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:04.225046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-f83560b1-7501-4dba-ab69-f7ea63afe647 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:04.226318 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f83560b1-7501-4dba-ab69-f7ea63afe647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:04.287077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:04.304178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1259/2527] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:01:04.305743 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3da34-2985-44ce-bce2-344320c4741d None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:04.306036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3da34-2985-44ce-bce2-344320c4741d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:04.306317 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3da34-2985-44ce-bce2-344320c4741d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:04.325993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f1a3da34-2985-44ce-bce2-344320c4741d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:04.327136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f1a3da34-2985-44ce-bce2-344320c4741d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:04.335334 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1269/2528] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:01:04.339605 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83560b1-7501-4dba-ab69-f7ea63afe647 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:04.350484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83560b1-7501-4dba-ab69-f7ea63afe647 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:04.356618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c248bba-f3e1-4d94-8b19-1c863ce22641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:04.357465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c248bba-f3e1-4d94-8b19-1c863ce22641 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:04.357755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c248bba-f3e1-4d94-8b19-1c863ce22641 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:04.358120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c248bba-f3e1-4d94-8b19-1c863ce22641 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:04.358916 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c248bba-f3e1-4d94-8b19-1c863ce22641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:01:04.359269 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c248bba-f3e1-4d94-8b19-1c863ce22641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:04.361556 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c248bba-f3e1-4d94-8b19-1c863ce22641 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:04.374685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1260/2529] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:01:04.379734 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3da34-2985-44ce-bce2-344320c4741d None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:04.389543 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3da34-2985-44ce-bce2-344320c4741d None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:04.394346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0eb8e3-8176-4883-9955-b741abc7edb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:04.395027 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0eb8e3-8176-4883-9955-b741abc7edb9 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:04.395315 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0eb8e3-8176-4883-9955-b741abc7edb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:04.395584 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0eb8e3-8176-4883-9955-b741abc7edb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:04.396314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0eb8e3-8176-4883-9955-b741abc7edb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:04.398248 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0eb8e3-8176-4883-9955-b741abc7edb9 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:04.426329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1270/2530] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:01:04 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:01:04.430397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c248bba-f3e1-4d94-8b19-1c863ce22641 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:04.440791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c248bba-f3e1-4d94-8b19-1c863ce22641 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:04.502463 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd8db6-fb3d-4e70-9e26-b95d608d7e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:04.503289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd8db6-fb3d-4e70-9e26-b95d608d7e8b admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:04.503544 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd8db6-fb3d-4e70-9e26-b95d608d7e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:04.503796 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbd8db6-fb3d-4e70-9e26-b95d608d7e8b admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:04.504525 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd8db6-fb3d-4e70-9e26-b95d608d7e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:04.506482 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbd8db6-fb3d-4e70-9e26-b95d608d7e8b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:04.527672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1261/2531] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:01:04 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:01:04.532380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0eb8e3-8176-4883-9955-b741abc7edb9 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:04.543033 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0eb8e3-8176-4883-9955-b741abc7edb9 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:04.547694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec45cc-2eef-4fd0-8cef-59f141331383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:04.548390 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec45cc-2eef-4fd0-8cef-59f141331383 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:04.548681 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec45cc-2eef-4fd0-8cef-59f141331383 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:04.548962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec45cc-2eef-4fd0-8cef-59f141331383 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:04.549718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec45cc-2eef-4fd0-8cef-59f141331383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:04.551569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec45cc-2eef-4fd0-8cef-59f141331383 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:04.562289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1271/2532] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:01:04.566402 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd8db6-fb3d-4e70-9e26-b95d608d7e8b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:04.577340 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbd8db6-fb3d-4e70-9e26-b95d608d7e8b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:04.581879 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754ca83-c83f-4874-a2f6-e1835a98a839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:04.582531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754ca83-c83f-4874-a2f6-e1835a98a839 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:04.582841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754ca83-c83f-4874-a2f6-e1835a98a839 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:04.583121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754ca83-c83f-4874-a2f6-e1835a98a839 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:04.583852 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754ca83-c83f-4874-a2f6-e1835a98a839 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:04.585469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754ca83-c83f-4874-a2f6-e1835a98a839 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:04.598151 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1262/2533] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:01:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 18:01:04.602489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec45cc-2eef-4fd0-8cef-59f141331383 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:04.612738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec45cc-2eef-4fd0-8cef-59f141331383 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:04.618029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b02949-74ac-43bd-b5bb-eeb7a07a7563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:04.618915 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b02949-74ac-43bd-b5bb-eeb7a07a7563 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:04.619196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b02949-74ac-43bd-b5bb-eeb7a07a7563 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:04.619473 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b02949-74ac-43bd-b5bb-eeb7a07a7563 admin admin] PATH_INFO: `/v3/projects/0b5f48c9a0474ac3be9913f056f0d35c/users/5416a5bf119b4e2293d67c68d43cad64/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:04.634506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b02949-74ac-43bd-b5bb-eeb7a07a7563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b5f48c9a0474ac3be9913f056f0d35c, user_id=5416a5bf119b4e2293d67c68d43cad64, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:04.637209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b02949-74ac-43bd-b5bb-eeb7a07a7563 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:04.651365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1272/2534] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:01:04 2022] PUT /identity/v3/projects/0b5f48c9a0474ac3be9913f056f0d35c/users/5416a5bf119b4e2293d67c68d43cad64/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:04.662747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82d595-cdad-4253-8036-2cbdf435d4cb None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:04.663109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82d595-cdad-4253-8036-2cbdf435d4cb None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:04.663464 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82d595-cdad-4253-8036-2cbdf435d4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:04.691015 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-9d82d595-cdad-4253-8036-2cbdf435d4cb None None] MFA Rules not processed for user `5416a5bf119b4e2293d67c68d43cad64`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:04.692413 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9d82d595-cdad-4253-8036-2cbdf435d4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:04.759168 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1263/2535] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:01:04.764309 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b02949-74ac-43bd-b5bb-eeb7a07a7563 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:04.803264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:04.843540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b02949-74ac-43bd-b5bb-eeb7a07a7563 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:04.849502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500ae82-fcdf-49f8-80e6-bb1e256062e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:04.850239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500ae82-fcdf-49f8-80e6-bb1e256062e9 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:04.850528 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500ae82-fcdf-49f8-80e6-bb1e256062e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:04.850886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500ae82-fcdf-49f8-80e6-bb1e256062e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:04.851587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500ae82-fcdf-49f8-80e6-bb1e256062e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:04.853576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500ae82-fcdf-49f8-80e6-bb1e256062e9 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:04.884547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1273/2536] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:01:04 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:01:04.890509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82d595-cdad-4253-8036-2cbdf435d4cb None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:04.903095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82d595-cdad-4253-8036-2cbdf435d4cb None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:04.969592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:04.979450 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c2f22-c228-410c-8f2d-3546b5d6e4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:04.980644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c2f22-c228-410c-8f2d-3546b5d6e4a5 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:04.981694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c2f22-c228-410c-8f2d-3546b5d6e4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:04.981694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6c2f22-c228-410c-8f2d-3546b5d6e4a5 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:04.982441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c2f22-c228-410c-8f2d-3546b5d6e4a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:04.984609 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6c2f22-c228-410c-8f2d-3546b5d6e4a5 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:05.014187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1264/2537] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:01:04 2022] POST /identity/v3/users => generated 301 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:01:05.015627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500ae82-fcdf-49f8-80e6-bb1e256062e9 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:05.026722 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500ae82-fcdf-49f8-80e6-bb1e256062e9 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:05.031740 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8997ed2-a38b-4729-aeeb-40081a6debaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:05.032460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8997ed2-a38b-4729-aeeb-40081a6debaf admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:05.032745 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8997ed2-a38b-4729-aeeb-40081a6debaf admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:05.033025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8997ed2-a38b-4729-aeeb-40081a6debaf admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:05.033760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8997ed2-a38b-4729-aeeb-40081a6debaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:05.035371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8997ed2-a38b-4729-aeeb-40081a6debaf admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:05.051850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1274/2538] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 18:01:05.053904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c2f22-c228-410c-8f2d-3546b5d6e4a5 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:05.064975 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6c2f22-c228-410c-8f2d-3546b5d6e4a5 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:05.069710 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a4e77-6e8d-45e5-8e0a-a7520db5c23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:05.070606 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4e77-6e8d-45e5-8e0a-a7520db5c23a admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:05.070890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4e77-6e8d-45e5-8e0a-a7520db5c23a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:05.071168 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4a4e77-6e8d-45e5-8e0a-a7520db5c23a admin admin] PATH_INFO: `/v3/projects/5cbf189271884f0cb2aa22216ae0cc43/users/b7c317a6bdda4535aff9ec9540ddc439/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:05.087066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a4e77-6e8d-45e5-8e0a-a7520db5c23a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbf189271884f0cb2aa22216ae0cc43, user_id=b7c317a6bdda4535aff9ec9540ddc439, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:05.089807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4a4e77-6e8d-45e5-8e0a-a7520db5c23a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:05.105035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1265/2539] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:01:05 2022] PUT /identity/v3/projects/5cbf189271884f0cb2aa22216ae0cc43/users/b7c317a6bdda4535aff9ec9540ddc439/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:05.109001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8997ed2-a38b-4729-aeeb-40081a6debaf admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:05.119354 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8997ed2-a38b-4729-aeeb-40081a6debaf admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:05.186098 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f495d887-dddd-4d42-97e7-a955b5dce80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:05.186792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495d887-dddd-4d42-97e7-a955b5dce80e admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:05.187083 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495d887-dddd-4d42-97e7-a955b5dce80e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:05.187366 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f495d887-dddd-4d42-97e7-a955b5dce80e admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:05.188139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495d887-dddd-4d42-97e7-a955b5dce80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:05.189742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f495d887-dddd-4d42-97e7-a955b5dce80e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:05.201531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1275/2540] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:01:05.205636 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a4e77-6e8d-45e5-8e0a-a7520db5c23a admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:05.216907 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4a4e77-6e8d-45e5-8e0a-a7520db5c23a admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:05.224474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77209d-1f3b-40fc-8834-ed4a4d1a2bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:05.225466 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77209d-1f3b-40fc-8834-ed4a4d1a2bcc admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:05.225823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77209d-1f3b-40fc-8834-ed4a4d1a2bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:05.226178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe77209d-1f3b-40fc-8834-ed4a4d1a2bcc admin admin] PATH_INFO: `/v3/projects/5cbf189271884f0cb2aa22216ae0cc43/users/b7c317a6bdda4535aff9ec9540ddc439/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:05.228700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77209d-1f3b-40fc-8834-ed4a4d1a2bcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cbf189271884f0cb2aa22216ae0cc43, user_id=b7c317a6bdda4535aff9ec9540ddc439, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:05.232264 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe77209d-1f3b-40fc-8834-ed4a4d1a2bcc admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:05.245850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1266/2541] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:01:05 2022] PUT /identity/v3/projects/5cbf189271884f0cb2aa22216ae0cc43/users/b7c317a6bdda4535aff9ec9540ddc439/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:05.257585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5859213-75f7-4c2a-95b1-cb3c7c17420a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:05.257878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5859213-75f7-4c2a-95b1-cb3c7c17420a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:05.258164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5859213-75f7-4c2a-95b1-cb3c7c17420a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:05.288432 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-e5859213-75f7-4c2a-95b1-cb3c7c17420a None None] MFA Rules not processed for user `b7c317a6bdda4535aff9ec9540ddc439`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:05.289671 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e5859213-75f7-4c2a-95b1-cb3c7c17420a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:05.381032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1276/2542] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:05 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:01:05.391822 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77209d-1f3b-40fc-8834-ed4a4d1a2bcc admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:05.394130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-fe77209d-1f3b-40fc-8834-ed4a4d1a2bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:05.460292 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe77209d-1f3b-40fc-8834-ed4a4d1a2bcc admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:05.464372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0b501-72fb-43bc-836d-7d5f306b76be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:05.465286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0b501-72fb-43bc-836d-7d5f306b76be service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:05.465799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0b501-72fb-43bc-836d-7d5f306b76be service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:05.466271 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0b501-72fb-43bc-836d-7d5f306b76be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:05.478312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0b501-72fb-43bc-836d-7d5f306b76be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:05.480806 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0b501-72fb-43bc-836d-7d5f306b76be service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:05.493669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1267/2543] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:01:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:01:05.633418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5859213-75f7-4c2a-95b1-cb3c7c17420a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:05.662795 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:05.704814 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5859213-75f7-4c2a-95b1-cb3c7c17420a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:05.710157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c37f59-5fa9-4a41-b652-ea2b3c129b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:05.711229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c37f59-5fa9-4a41-b652-ea2b3c129b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:05.711595 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c37f59-5fa9-4a41-b652-ea2b3c129b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:05.711975 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c37f59-5fa9-4a41-b652-ea2b3c129b11 admin admin] PATH_INFO: `/v3/users/5416a5bf119b4e2293d67c68d43cad64` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:05.713323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c37f59-5fa9-4a41-b652-ea2b3c129b11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5416a5bf119b4e2293d67c68d43cad64)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:05.716061 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c37f59-5fa9-4a41-b652-ea2b3c129b11 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:05.772092 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f6c37f59-5fa9-4a41-b652-ea2b3c129b11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5416a5bf119b4e2293d67c68d43cad64'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:05.786757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f6c37f59-5fa9-4a41-b652-ea2b3c129b11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5416a5bf119b4e2293d67c68d43cad64'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:05.797358 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f6c37f59-5fa9-4a41-b652-ea2b3c129b11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5416a5bf119b4e2293d67c68d43cad64'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:05.798190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f6c37f59-5fa9-4a41-b652-ea2b3c129b11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5416a5bf119b4e2293d67c68d43cad64'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:05.810369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f6c37f59-5fa9-4a41-b652-ea2b3c129b11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5416a5bf119b4e2293d67c68d43cad64'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:05.818294 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1277/2544] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:01:05 2022] DELETE /identity/v3/users/5416a5bf119b4e2293d67c68d43cad64 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:05.822602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1e7a2-3947-4923-8a63-c8852656d1a6 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:05.822901 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1e7a2-3947-4923-8a63-c8852656d1a6 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:05.823184 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1e7a2-3947-4923-8a63-c8852656d1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:05.848001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-3ab1e7a2-3947-4923-8a63-c8852656d1a6 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:05.849321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3ab1e7a2-3947-4923-8a63-c8852656d1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:05.915739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1268/2545] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:01:05.926421 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c37f59-5fa9-4a41-b652-ea2b3c129b11 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:05.928004 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f6c37f59-5fa9-4a41-b652-ea2b3c129b11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:05.991809 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c37f59-5fa9-4a41-b652-ea2b3c129b11 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:05.995877 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c5ca0-dc42-4915-9e6d-ea8744098fb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:05.996579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c5ca0-dc42-4915-9e6d-ea8744098fb3 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:05.997045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c5ca0-dc42-4915-9e6d-ea8744098fb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:05.997379 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c5ca0-dc42-4915-9e6d-ea8744098fb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:06.008093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c5ca0-dc42-4915-9e6d-ea8744098fb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:06.009565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9c5ca0-dc42-4915-9e6d-ea8744098fb3 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:06.022866 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1278/2546] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:01:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 18:01:06.345762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1e7a2-3947-4923-8a63-c8852656d1a6 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:06.348937 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3ab1e7a2-3947-4923-8a63-c8852656d1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:06.367520 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1e7a2-3947-4923-8a63-c8852656d1a6 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:06.388404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:06.389303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:06.389634 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:06.389874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] PATH_INFO: `/v3/projects/0b5f48c9a0474ac3be9913f056f0d35c` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:06.391148 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b5f48c9a0474ac3be9913f056f0d35c)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:06.393304 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:06.458193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:06.462126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b5f48c9a0474ac3be9913f056f0d35c', 'request_id': 'req-de0149ad-6608-47db-bb96-ae4893294faa'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:06.474992 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b5f48c9a0474ac3be9913f056f0d35c', 'request_id': 'req-de0149ad-6608-47db-bb96-ae4893294faa'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:06.485768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b5f48c9a0474ac3be9913f056f0d35c', 'request_id': 'req-de0149ad-6608-47db-bb96-ae4893294faa'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:06.500963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] The token cache is being invalidate because project 0b5f48c9a0474ac3be9913f056f0d35c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:01:06.501253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:06.501950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:06.506980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1269/2547] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:01:06 2022] DELETE /identity/v3/projects/0b5f48c9a0474ac3be9913f056f0d35c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:06.508786 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c5ca0-dc42-4915-9e6d-ea8744098fb3 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:06.521623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c5ca0-dc42-4915-9e6d-ea8744098fb3 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:06.589337 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:06.610958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8820fb-272c-4853-916d-fd5b49a2b407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:06.612051 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8820fb-272c-4853-916d-fd5b49a2b407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:06.612340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8820fb-272c-4853-916d-fd5b49a2b407 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:06.612672 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8820fb-272c-4853-916d-fd5b49a2b407 admin admin] PATH_INFO: `/v3/users/b7c317a6bdda4535aff9ec9540ddc439` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:06.613917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8820fb-272c-4853-916d-fd5b49a2b407 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7c317a6bdda4535aff9ec9540ddc439)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:06.615861 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8820fb-272c-4853-916d-fd5b49a2b407 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:06.684812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ea8820fb-272c-4853-916d-fd5b49a2b407 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7c317a6bdda4535aff9ec9540ddc439'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:06.705638 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ea8820fb-272c-4853-916d-fd5b49a2b407 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7c317a6bdda4535aff9ec9540ddc439'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:06.716157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ea8820fb-272c-4853-916d-fd5b49a2b407 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7c317a6bdda4535aff9ec9540ddc439'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:06.716857 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ea8820fb-272c-4853-916d-fd5b49a2b407 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7c317a6bdda4535aff9ec9540ddc439'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:06.730007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ea8820fb-272c-4853-916d-fd5b49a2b407 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7c317a6bdda4535aff9ec9540ddc439'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:06.737810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1279/2548] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:01:06 2022] DELETE /identity/v3/users/b7c317a6bdda4535aff9ec9540ddc439 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:07.025942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:07.027496 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:07.116211 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0149ad-6608-47db-bb96-ae4893294faa admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:07.200318 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8820fb-272c-4853-916d-fd5b49a2b407 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:07.202107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ea8820fb-272c-4853-916d-fd5b49a2b407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:07.204471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ab23a9-8829-4a6c-b8b0-2e6097343bd2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:07.205218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab23a9-8829-4a6c-b8b0-2e6097343bd2 service placement] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:07.205435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab23a9-8829-4a6c-b8b0-2e6097343bd2 service placement] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:07.205730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ab23a9-8829-4a6c-b8b0-2e6097343bd2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:07.208370 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-20ab23a9-8829-4a6c-b8b0-2e6097343bd2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:07.237234 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab23a9-8829-4a6c-b8b0-2e6097343bd2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:07.239278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ab23a9-8829-4a6c-b8b0-2e6097343bd2 service placement] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:07.255443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1270/2549] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:01:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:01:07.330130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8820fb-272c-4853-916d-fd5b49a2b407 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:07.336213 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df17ef-6c68-41ce-b48a-b8b76340dfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:07.337114 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df17ef-6c68-41ce-b48a-b8b76340dfb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:07.337402 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df17ef-6c68-41ce-b48a-b8b76340dfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:07.337683 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df17ef-6c68-41ce-b48a-b8b76340dfb2 admin admin] PATH_INFO: `/v3/projects/5cbf189271884f0cb2aa22216ae0cc43` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:07.339706 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df17ef-6c68-41ce-b48a-b8b76340dfb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cbf189271884f0cb2aa22216ae0cc43)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:07.341669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df17ef-6c68-41ce-b48a-b8b76340dfb2 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:07.392578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-73df17ef-6c68-41ce-b48a-b8b76340dfb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cbf189271884f0cb2aa22216ae0cc43', 'request_id': 'req-73df17ef-6c68-41ce-b48a-b8b76340dfb2'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:07.401510 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-73df17ef-6c68-41ce-b48a-b8b76340dfb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cbf189271884f0cb2aa22216ae0cc43', 'request_id': 'req-73df17ef-6c68-41ce-b48a-b8b76340dfb2'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:07.408183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-73df17ef-6c68-41ce-b48a-b8b76340dfb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cbf189271884f0cb2aa22216ae0cc43', 'request_id': 'req-73df17ef-6c68-41ce-b48a-b8b76340dfb2'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:07.415119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:07.424143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-73df17ef-6c68-41ce-b48a-b8b76340dfb2 admin admin] The token cache is being invalidate because project 5cbf189271884f0cb2aa22216ae0cc43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:01:07.425087 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-73df17ef-6c68-41ce-b48a-b8b76340dfb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:07.426350 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-73df17ef-6c68-41ce-b48a-b8b76340dfb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:07.428348 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1280/2550] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:01:07 2022] DELETE /identity/v3/projects/5cbf189271884f0cb2aa22216ae0cc43 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:07.435189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e27a9-0963-4b4e-a7ab-00649f9ef74c None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:07.437930 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e27a9-0963-4b4e-a7ab-00649f9ef74c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:07.438440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2e27a9-0963-4b4e-a7ab-00649f9ef74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:07.465739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-dd2e27a9-0963-4b4e-a7ab-00649f9ef74c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:07.467121 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-dd2e27a9-0963-4b4e-a7ab-00649f9ef74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:07.527186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1271/2551] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:01:07.535238 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dd61c-8219-49de-bae9-430cd1c6eed6 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:07.535527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dd61c-8219-49de-bae9-430cd1c6eed6 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:07.535802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495dd61c-8219-49de-bae9-430cd1c6eed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:07.554731 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-495dd61c-8219-49de-bae9-430cd1c6eed6 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:07.556265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-495dd61c-8219-49de-bae9-430cd1c6eed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:07.564495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1281/2552] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:01:07.570064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e27a9-0963-4b4e-a7ab-00649f9ef74c None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:07.579799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2e27a9-0963-4b4e-a7ab-00649f9ef74c None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:07.584952 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1347c3f-b831-4fab-af02-6b420cee3d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:07.585752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1347c3f-b831-4fab-af02-6b420cee3d10 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:07.586045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1347c3f-b831-4fab-af02-6b420cee3d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:07.586360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1347c3f-b831-4fab-af02-6b420cee3d10 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:07.587107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1347c3f-b831-4fab-af02-6b420cee3d10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:01:07.587421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1347c3f-b831-4fab-af02-6b420cee3d10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:07.589638 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1347c3f-b831-4fab-af02-6b420cee3d10 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:07.591355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:07.609773 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1272/2553] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:01:07.611224 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dd61c-8219-49de-bae9-430cd1c6eed6 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:07.620582 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495dd61c-8219-49de-bae9-430cd1c6eed6 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:07.625284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7609d3b-9ead-498e-b5a5-696183adea9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:07.625952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7609d3b-9ead-498e-b5a5-696183adea9e admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:07.626235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7609d3b-9ead-498e-b5a5-696183adea9e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:07.626511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7609d3b-9ead-498e-b5a5-696183adea9e admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:07.627202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7609d3b-9ead-498e-b5a5-696183adea9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:07.629146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7609d3b-9ead-498e-b5a5-696183adea9e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:07.662799 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1282/2554] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:01:07 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:01:07.667057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1347c3f-b831-4fab-af02-6b420cee3d10 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:07.679767 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1347c3f-b831-4fab-af02-6b420cee3d10 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:07.743788 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7dcec5-4a3c-4f16-854e-fd1d2d96e191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:07.744516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7dcec5-4a3c-4f16-854e-fd1d2d96e191 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:07.744801 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7dcec5-4a3c-4f16-854e-fd1d2d96e191 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:07.745080 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7dcec5-4a3c-4f16-854e-fd1d2d96e191 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:07.745775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7dcec5-4a3c-4f16-854e-fd1d2d96e191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:07.747687 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7dcec5-4a3c-4f16-854e-fd1d2d96e191 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:07.771150 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1273/2555] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:01:07 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:01:07.772652 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7609d3b-9ead-498e-b5a5-696183adea9e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:07.783538 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7609d3b-9ead-498e-b5a5-696183adea9e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:07.792167 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171a635-89c2-46c9-a547-9c321f13e832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:07.792661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171a635-89c2-46c9-a547-9c321f13e832 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:07.792951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171a635-89c2-46c9-a547-9c321f13e832 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:07.793237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2171a635-89c2-46c9-a547-9c321f13e832 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:07.793979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171a635-89c2-46c9-a547-9c321f13e832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:07.795575 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2171a635-89c2-46c9-a547-9c321f13e832 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:07.812497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1283/2556] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:01:07.816692 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7dcec5-4a3c-4f16-854e-fd1d2d96e191 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:07.828024 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7dcec5-4a3c-4f16-854e-fd1d2d96e191 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:07.832728 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27a7f0-790d-43ce-b7cf-3acc9172164b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:07.833356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27a7f0-790d-43ce-b7cf-3acc9172164b admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:07.833642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27a7f0-790d-43ce-b7cf-3acc9172164b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:07.833921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27a7f0-790d-43ce-b7cf-3acc9172164b admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:07.834774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27a7f0-790d-43ce-b7cf-3acc9172164b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:07.836413 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27a7f0-790d-43ce-b7cf-3acc9172164b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:07.848053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1274/2557] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:01:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 18:01:07.852157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171a635-89c2-46c9-a547-9c321f13e832 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:07.862316 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2171a635-89c2-46c9-a547-9c321f13e832 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:07.867067 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff47efc-bf73-43c1-a60c-51c6ccd8a994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:07.867974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff47efc-bf73-43c1-a60c-51c6ccd8a994 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:07.868265 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff47efc-bf73-43c1-a60c-51c6ccd8a994 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:07.868547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff47efc-bf73-43c1-a60c-51c6ccd8a994 admin admin] PATH_INFO: `/v3/projects/1f4e220af1bf47d6948d96dd4656bee4/users/f7182121b56e4aef82aa39b173702d3c/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:07.883826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff47efc-bf73-43c1-a60c-51c6ccd8a994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f4e220af1bf47d6948d96dd4656bee4, user_id=f7182121b56e4aef82aa39b173702d3c, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:07.887152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff47efc-bf73-43c1-a60c-51c6ccd8a994 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:07.900954 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1284/2558] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:01:07 2022] PUT /identity/v3/projects/1f4e220af1bf47d6948d96dd4656bee4/users/f7182121b56e4aef82aa39b173702d3c/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:07.912738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd403d3b-0b11-4f39-980a-d20491749b83 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:07.913075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd403d3b-0b11-4f39-980a-d20491749b83 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:07.913320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd403d3b-0b11-4f39-980a-d20491749b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:07.941869 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-bd403d3b-0b11-4f39-980a-d20491749b83 None None] MFA Rules not processed for user `f7182121b56e4aef82aa39b173702d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:07.943230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-bd403d3b-0b11-4f39-980a-d20491749b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:08.011447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1275/2559] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:07 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:01:08.022952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff47efc-bf73-43c1-a60c-51c6ccd8a994 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:08.024765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-bff47efc-bf73-43c1-a60c-51c6ccd8a994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:08.091533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff47efc-bf73-43c1-a60c-51c6ccd8a994 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:08.100407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0842e-0fd2-4f2b-b19c-0b6aa778ab99 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2b4bf2aa1744a46b6d2a1b5a136a0e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:08.101089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0842e-0fd2-4f2b-b19c-0b6aa778ab99 service glance] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:08.101372 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0842e-0fd2-4f2b-b19c-0b6aa778ab99 service glance] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:08.101649 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a0842e-0fd2-4f2b-b19c-0b6aa778ab99 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:08.111237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0842e-0fd2-4f2b-b19c-0b6aa778ab99 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:08.112718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a0842e-0fd2-4f2b-b19c-0b6aa778ab99 service glance] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:08.126129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1285/2560] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:01:08 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:01:10.684409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd403d3b-0b11-4f39-980a-d20491749b83 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:10.686004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-bd403d3b-0b11-4f39-980a-d20491749b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:10.753528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd403d3b-0b11-4f39-980a-d20491749b83 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:10.758117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a9cef-b1e3-4eec-bfd4-4af097726cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:10.758997 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a9cef-b1e3-4eec-bfd4-4af097726cf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:10.759283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a9cef-b1e3-4eec-bfd4-4af097726cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:10.759557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6a9cef-b1e3-4eec-bfd4-4af097726cf7 admin admin] PATH_INFO: `/v3/users/3e3fd98743554b0a8bf1cea0523ed4ae` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:10.760813 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a9cef-b1e3-4eec-bfd4-4af097726cf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e3fd98743554b0a8bf1cea0523ed4ae)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:10.762740 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6a9cef-b1e3-4eec-bfd4-4af097726cf7 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:10.765118 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:10.818595 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7f6a9cef-b1e3-4eec-bfd4-4af097726cf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e3fd98743554b0a8bf1cea0523ed4ae'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:10.831068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7f6a9cef-b1e3-4eec-bfd4-4af097726cf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e3fd98743554b0a8bf1cea0523ed4ae'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:10.831803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7f6a9cef-b1e3-4eec-bfd4-4af097726cf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e3fd98743554b0a8bf1cea0523ed4ae'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:10.839635 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7f6a9cef-b1e3-4eec-bfd4-4af097726cf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e3fd98743554b0a8bf1cea0523ed4ae'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:10.840322 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-7f6a9cef-b1e3-4eec-bfd4-4af097726cf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e3fd98743554b0a8bf1cea0523ed4ae'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:10.853475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1276/2561] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:01:10 2022] DELETE /identity/v3/users/3e3fd98743554b0a8bf1cea0523ed4ae => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:11.033185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0842e-0fd2-4f2b-b19c-0b6aa778ab99 service glance] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:11.047910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a0842e-0fd2-4f2b-b19c-0b6aa778ab99 service glance] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:11.111454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:11.112366 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:11.112650 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:11.112952 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] PATH_INFO: `/v3/projects/fa36825e7c014d48a7fe9375d9abdf7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:11.114124 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa36825e7c014d48a7fe9375d9abdf7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:11.116120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:11.156252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:11.168268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa36825e7c014d48a7fe9375d9abdf7d', 'request_id': 'req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.178590 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa36825e7c014d48a7fe9375d9abdf7d', 'request_id': 'req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.187795 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa36825e7c014d48a7fe9375d9abdf7d', 'request_id': 'req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.207298 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] The token cache is being invalidate because project fa36825e7c014d48a7fe9375d9abdf7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:01:11.207637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.208318 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.210507 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1286/2562] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:01:11 2022] DELETE /identity/v3/projects/fa36825e7c014d48a7fe9375d9abdf7d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:11.214460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a9cef-b1e3-4eec-bfd4-4af097726cf7 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:11.215875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7f6a9cef-b1e3-4eec-bfd4-4af097726cf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:11.228708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6a9cef-b1e3-4eec-bfd4-4af097726cf7 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:11.290874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fc123-a9d6-441f-905c-a56f54741fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:11.291842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fc123-a9d6-441f-905c-a56f54741fdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:11.292091 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fc123-a9d6-441f-905c-a56f54741fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:11.292374 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fc123-a9d6-441f-905c-a56f54741fdf admin admin] PATH_INFO: `/v3/users/79e6432ebc1c4eb496e5af7b8eabfab4` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:11.293608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fc123-a9d6-441f-905c-a56f54741fdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=79e6432ebc1c4eb496e5af7b8eabfab4)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:11.295794 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fc123-a9d6-441f-905c-a56f54741fdf admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:11.344712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3e4fc123-a9d6-441f-905c-a56f54741fdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79e6432ebc1c4eb496e5af7b8eabfab4'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.359294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3e4fc123-a9d6-441f-905c-a56f54741fdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79e6432ebc1c4eb496e5af7b8eabfab4'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.360066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3e4fc123-a9d6-441f-905c-a56f54741fdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79e6432ebc1c4eb496e5af7b8eabfab4'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.367306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3e4fc123-a9d6-441f-905c-a56f54741fdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79e6432ebc1c4eb496e5af7b8eabfab4'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.367987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-3e4fc123-a9d6-441f-905c-a56f54741fdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79e6432ebc1c4eb496e5af7b8eabfab4'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.376501 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:11.382057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1277/2563] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:01:11 2022] DELETE /identity/v3/users/79e6432ebc1c4eb496e5af7b8eabfab4 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:01:11.559695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:11.561498 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:11.576301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcc5ee-a240-4bd4-8932-fe844ba85b93 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:11.636709 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:11.637578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:11.637865 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:11.638166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] PATH_INFO: `/v3/projects/59e2d432a0c54303ae92759916a9fc37` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:11.639342 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=59e2d432a0c54303ae92759916a9fc37)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:11.641282 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:11.688705 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59e2d432a0c54303ae92759916a9fc37', 'request_id': 'req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.698781 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59e2d432a0c54303ae92759916a9fc37', 'request_id': 'req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.706237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59e2d432a0c54303ae92759916a9fc37', 'request_id': 'req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.717009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] The token cache is being invalidate because project 59e2d432a0c54303ae92759916a9fc37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:01:11.717295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.717947 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:11.719488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1287/2564] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:01:11 2022] DELETE /identity/v3/projects/59e2d432a0c54303ae92759916a9fc37 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:01:21.713738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=62,keystone:DELETE=11,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:48.390797 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fc123-a9d6-441f-905c-a56f54741fdf admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:48.392287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3e4fc123-a9d6-441f-905c-a56f54741fdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:48.453072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fc123-a9d6-441f-905c-a56f54741fdf admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:48.503878 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15c4da-beb7-4aaf-8721-a5e094e5c43d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:48.504564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15c4da-beb7-4aaf-8721-a5e094e5c43d service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:48.504853 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15c4da-beb7-4aaf-8721-a5e094e5c43d service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:48.505130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15c4da-beb7-4aaf-8721-a5e094e5c43d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:48.506928 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-dd15c4da-beb7-4aaf-8721-a5e094e5c43d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:48.587176 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15c4da-beb7-4aaf-8721-a5e094e5c43d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:48.588686 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd15c4da-beb7-4aaf-8721-a5e094e5c43d service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:48.600620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1278/2565] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:01:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:01:52.141090 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:52.142619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:52.155208 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fb3f44-ccb6-4fd4-b340-f3a01b9eee2a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:52.159459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd5a45-e069-442c-879d-86107b9152d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:52.160164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd5a45-e069-442c-879d-86107b9152d0 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:52.160458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd5a45-e069-442c-879d-86107b9152d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:52.160738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd5a45-e069-442c-879d-86107b9152d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:52.162554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-91fd5a45-e069-442c-879d-86107b9152d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:52.177347 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd5a45-e069-442c-879d-86107b9152d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:52.178796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd5a45-e069-442c-879d-86107b9152d0 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:52.190455 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1288/2566] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:01:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:01:56.097812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15c4da-beb7-4aaf-8721-a5e094e5c43d service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:56.099621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-dd15c4da-beb7-4aaf-8721-a5e094e5c43d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:56.177143 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:56.181939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15c4da-beb7-4aaf-8721-a5e094e5c43d service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:56.187257 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1b97c-5f5a-45e0-b9b6-5e64c18d5805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:56.188183 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1b97c-5f5a-45e0-b9b6-5e64c18d5805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:56.188470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1b97c-5f5a-45e0-b9b6-5e64c18d5805 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:56.188872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced1b97c-5f5a-45e0-b9b6-5e64c18d5805 admin admin] PATH_INFO: `/v3/users/f7182121b56e4aef82aa39b173702d3c` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:56.190083 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1b97c-5f5a-45e0-b9b6-5e64c18d5805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7182121b56e4aef82aa39b173702d3c)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:56.192708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced1b97c-5f5a-45e0-b9b6-5e64c18d5805 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:56.240934 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ced1b97c-5f5a-45e0-b9b6-5e64c18d5805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7182121b56e4aef82aa39b173702d3c'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:56.254370 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ced1b97c-5f5a-45e0-b9b6-5e64c18d5805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7182121b56e4aef82aa39b173702d3c'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:56.255005 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ced1b97c-5f5a-45e0-b9b6-5e64c18d5805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7182121b56e4aef82aa39b173702d3c'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:56.262096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ced1b97c-5f5a-45e0-b9b6-5e64c18d5805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7182121b56e4aef82aa39b173702d3c'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:56.263039 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ced1b97c-5f5a-45e0-b9b6-5e64c18d5805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7182121b56e4aef82aa39b173702d3c'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:56.275336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1279/2567] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:01:56 2022] DELETE /identity/v3/users/f7182121b56e4aef82aa39b173702d3c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:56.280142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86bb69-3d3e-46d9-b54f-699a0f929994 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:56.280695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86bb69-3d3e-46d9-b54f-699a0f929994 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:56.280695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86bb69-3d3e-46d9-b54f-699a0f929994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:56.298666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-6c86bb69-3d3e-46d9-b54f-699a0f929994 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:56.299892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6c86bb69-3d3e-46d9-b54f-699a0f929994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:56.364669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1289/2568] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:01:56.374437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1b97c-5f5a-45e0-b9b6-5e64c18d5805 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:56.375869 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ced1b97c-5f5a-45e0-b9b6-5e64c18d5805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:56.438412 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced1b97c-5f5a-45e0-b9b6-5e64c18d5805 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:56.442125 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194a408-34a9-4771-be71-33d2152ee3ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:56.442782 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194a408-34a9-4771-be71-33d2152ee3ab service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:56.443066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194a408-34a9-4771-be71-33d2152ee3ab service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:56.443348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194a408-34a9-4771-be71-33d2152ee3ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:56.452817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194a408-34a9-4771-be71-33d2152ee3ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:56.454246 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194a408-34a9-4771-be71-33d2152ee3ab service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:56.463744 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1280/2569] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:01:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:01:56.887457 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86bb69-3d3e-46d9-b54f-699a0f929994 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:56.889524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6c86bb69-3d3e-46d9-b54f-699a0f929994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:56.909767 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86bb69-3d3e-46d9-b54f-699a0f929994 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:56.915904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6988791c-4f0a-4426-a8d0-e5b8279d10c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:56.916919 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988791c-4f0a-4426-a8d0-e5b8279d10c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:56.917214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988791c-4f0a-4426-a8d0-e5b8279d10c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:56.917497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6988791c-4f0a-4426-a8d0-e5b8279d10c1 admin admin] PATH_INFO: `/v3/projects/1f4e220af1bf47d6948d96dd4656bee4` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:56.918825 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988791c-4f0a-4426-a8d0-e5b8279d10c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f4e220af1bf47d6948d96dd4656bee4)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:56.921063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6988791c-4f0a-4426-a8d0-e5b8279d10c1 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:57.032768 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-6988791c-4f0a-4426-a8d0-e5b8279d10c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f4e220af1bf47d6948d96dd4656bee4', 'request_id': 'req-6988791c-4f0a-4426-a8d0-e5b8279d10c1'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:57.047386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-6988791c-4f0a-4426-a8d0-e5b8279d10c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f4e220af1bf47d6948d96dd4656bee4', 'request_id': 'req-6988791c-4f0a-4426-a8d0-e5b8279d10c1'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:57.061816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-6988791c-4f0a-4426-a8d0-e5b8279d10c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f4e220af1bf47d6948d96dd4656bee4', 'request_id': 'req-6988791c-4f0a-4426-a8d0-e5b8279d10c1'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:57.084221 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-6988791c-4f0a-4426-a8d0-e5b8279d10c1 admin admin] The token cache is being invalidate because project 1f4e220af1bf47d6948d96dd4656bee4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:01:57.084708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-6988791c-4f0a-4426-a8d0-e5b8279d10c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:57.087254 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-6988791c-4f0a-4426-a8d0-e5b8279d10c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:01:57.089887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1290/2570] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:01:56 2022] DELETE /identity/v3/projects/1f4e220af1bf47d6948d96dd4656bee4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:57.096110 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ae054-407c-4036-b8df-bfb054b8eff9 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:57.096321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ae054-407c-4036-b8df-bfb054b8eff9 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:57.096609 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ae054-407c-4036-b8df-bfb054b8eff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:57.122844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-274ae054-407c-4036-b8df-bfb054b8eff9 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:57.124163 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-274ae054-407c-4036-b8df-bfb054b8eff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:57.202350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1281/2571] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:01:57.210428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318d4c0-6edc-484a-9852-79ac3d0757b1 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:57.210724 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318d4c0-6edc-484a-9852-79ac3d0757b1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:57.211016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318d4c0-6edc-484a-9852-79ac3d0757b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:57.229765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-d318d4c0-6edc-484a-9852-79ac3d0757b1 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:57.231009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d318d4c0-6edc-484a-9852-79ac3d0757b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:57.245881 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1291/2572] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:01:57.249786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ae054-407c-4036-b8df-bfb054b8eff9 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:57.261146 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ae054-407c-4036-b8df-bfb054b8eff9 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:57.267560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5976ce-9763-413a-8c96-59fcf5905c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:57.268530 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5976ce-9763-413a-8c96-59fcf5905c8f admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:57.268921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5976ce-9763-413a-8c96-59fcf5905c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:57.269508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5976ce-9763-413a-8c96-59fcf5905c8f admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:57.270440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5976ce-9763-413a-8c96-59fcf5905c8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:01:57.270778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5976ce-9763-413a-8c96-59fcf5905c8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:57.273359 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5976ce-9763-413a-8c96-59fcf5905c8f admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:57.287109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1282/2573] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:01:57.291159 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318d4c0-6edc-484a-9852-79ac3d0757b1 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:57.300698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318d4c0-6edc-484a-9852-79ac3d0757b1 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:57.305541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101336ff-6cac-497d-bad1-f36199efa13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:57.306207 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101336ff-6cac-497d-bad1-f36199efa13c admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:57.306496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101336ff-6cac-497d-bad1-f36199efa13c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:57.306768 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101336ff-6cac-497d-bad1-f36199efa13c admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:57.307457 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101336ff-6cac-497d-bad1-f36199efa13c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:57.309371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101336ff-6cac-497d-bad1-f36199efa13c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:57.340364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1292/2574] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:01:57 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:01:57.341630 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5976ce-9763-413a-8c96-59fcf5905c8f admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:57.351838 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5976ce-9763-413a-8c96-59fcf5905c8f admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:57.355514 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:57.414341 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b6b7e-4786-4c50-a161-3c1e8cc57c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:57.415016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b6b7e-4786-4c50-a161-3c1e8cc57c6d admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:57.415297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b6b7e-4786-4c50-a161-3c1e8cc57c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:57.415573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b6b7e-4786-4c50-a161-3c1e8cc57c6d admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:57.416301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b6b7e-4786-4c50-a161-3c1e8cc57c6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:57.418203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b6b7e-4786-4c50-a161-3c1e8cc57c6d admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:57.439727 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1283/2575] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:01:57 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:01:57.443975 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101336ff-6cac-497d-bad1-f36199efa13c admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:57.455002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101336ff-6cac-497d-bad1-f36199efa13c admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:57.459607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45970bd7-5ab7-4896-b90c-b553fadb0ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:57.460375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970bd7-5ab7-4896-b90c-b553fadb0ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:57.460732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970bd7-5ab7-4896-b90c-b553fadb0ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:57.461077 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45970bd7-5ab7-4896-b90c-b553fadb0ab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:57.461871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45970bd7-5ab7-4896-b90c-b553fadb0ab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:57.463510 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45970bd7-5ab7-4896-b90c-b553fadb0ab1 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:57.493733 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1293/2576] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:01:57.498375 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b6b7e-4786-4c50-a161-3c1e8cc57c6d admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:57.508528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b6b7e-4786-4c50-a161-3c1e8cc57c6d admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:57.513484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0703d608-9e9f-4a12-b0c4-15ef35c265ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:57.514157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703d608-9e9f-4a12-b0c4-15ef35c265ea admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:57.514441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703d608-9e9f-4a12-b0c4-15ef35c265ea admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:57.514718 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0703d608-9e9f-4a12-b0c4-15ef35c265ea admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:57.515476 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703d608-9e9f-4a12-b0c4-15ef35c265ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:57.517124 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0703d608-9e9f-4a12-b0c4-15ef35c265ea admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:57.527360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1284/2577] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:01:57.531541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45970bd7-5ab7-4896-b90c-b553fadb0ab1 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:57.541521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45970bd7-5ab7-4896-b90c-b553fadb0ab1 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:57.546666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74909d-1bc5-47f8-a3bd-5a27b6dc481a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:57.547556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74909d-1bc5-47f8-a3bd-5a27b6dc481a admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:57.547838 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74909d-1bc5-47f8-a3bd-5a27b6dc481a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:57.548147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74909d-1bc5-47f8-a3bd-5a27b6dc481a admin admin] PATH_INFO: `/v3/projects/6559979e7e3b443a9e1633209bf113bb/users/d6e9a4bfb4184805b9088d6eab6f11c1/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:57.565239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74909d-1bc5-47f8-a3bd-5a27b6dc481a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6559979e7e3b443a9e1633209bf113bb, user_id=d6e9a4bfb4184805b9088d6eab6f11c1, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:57.567896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b74909d-1bc5-47f8-a3bd-5a27b6dc481a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:57.583488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1294/2578] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:01:57 2022] PUT /identity/v3/projects/6559979e7e3b443a9e1633209bf113bb/users/d6e9a4bfb4184805b9088d6eab6f11c1/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:01:57.593910 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ced75a-a1ba-4d45-a15e-e03a4d1f0ffe None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:57.594198 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ced75a-a1ba-4d45-a15e-e03a4d1f0ffe None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:57.594526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ced75a-a1ba-4d45-a15e-e03a4d1f0ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:57.623418 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-37ced75a-a1ba-4d45-a15e-e03a4d1f0ffe None None] MFA Rules not processed for user `d6e9a4bfb4184805b9088d6eab6f11c1`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:01:57.624761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-37ced75a-a1ba-4d45-a15e-e03a4d1f0ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:57.692745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1285/2579] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:01:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:01:57.705122 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74909d-1bc5-47f8-a3bd-5a27b6dc481a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:01:57.707012 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1b74909d-1bc5-47f8-a3bd-5a27b6dc481a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:01:57.717844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:01:57.772169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74909d-1bc5-47f8-a3bd-5a27b6dc481a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:01:57.776242 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9452c0fe-da87-47c0-a780-083017a8759a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:01:57.776544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452c0fe-da87-47c0-a780-083017a8759a service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:01:57.776831 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452c0fe-da87-47c0-a780-083017a8759a service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:01:57.777142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452c0fe-da87-47c0-a780-083017a8759a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:01:57.786489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452c0fe-da87-47c0-a780-083017a8759a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:01:57.788025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9452c0fe-da87-47c0-a780-083017a8759a service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:01:57.797545 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1295/2580] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:01:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:02:00.354136 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ced75a-a1ba-4d45-a15e-e03a4d1f0ffe None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:00.419909 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ced75a-a1ba-4d45-a15e-e03a4d1f0ffe None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:00.424494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef377b4-c637-4613-94a9-4d40765a0687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:00.425385 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef377b4-c637-4613-94a9-4d40765a0687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:00.425669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef377b4-c637-4613-94a9-4d40765a0687 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:00.425955 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef377b4-c637-4613-94a9-4d40765a0687 admin admin] PATH_INFO: `/v3/users/d6e9a4bfb4184805b9088d6eab6f11c1` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:00.427185 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef377b4-c637-4613-94a9-4d40765a0687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6e9a4bfb4184805b9088d6eab6f11c1)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:00.429158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef377b4-c637-4613-94a9-4d40765a0687 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:00.446167 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:02:00.478283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-6ef377b4-c637-4613-94a9-4d40765a0687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6e9a4bfb4184805b9088d6eab6f11c1'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:00.490446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-6ef377b4-c637-4613-94a9-4d40765a0687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6e9a4bfb4184805b9088d6eab6f11c1'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:00.491128 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-6ef377b4-c637-4613-94a9-4d40765a0687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6e9a4bfb4184805b9088d6eab6f11c1'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:00.498283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-6ef377b4-c637-4613-94a9-4d40765a0687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6e9a4bfb4184805b9088d6eab6f11c1'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:00.498896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-6ef377b4-c637-4613-94a9-4d40765a0687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6e9a4bfb4184805b9088d6eab6f11c1'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:00.511774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1286/2581] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:02:00 2022] DELETE /identity/v3/users/d6e9a4bfb4184805b9088d6eab6f11c1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:02:00.516303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea2f0e-1d36-4596-a39b-20d6b0700460 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:00.516796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea2f0e-1d36-4596-a39b-20d6b0700460 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:00.516910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea2f0e-1d36-4596-a39b-20d6b0700460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:00.534478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-23ea2f0e-1d36-4596-a39b-20d6b0700460 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:02:00.535679 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-23ea2f0e-1d36-4596-a39b-20d6b0700460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:00.595151 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1296/2582] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:02:00.604915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef377b4-c637-4613-94a9-4d40765a0687 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:00.606346 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6ef377b4-c637-4613-94a9-4d40765a0687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:00.667729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef377b4-c637-4613-94a9-4d40765a0687 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:00.671484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f577aee-73ca-4392-90fb-4da5a68d7339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:00.672119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f577aee-73ca-4392-90fb-4da5a68d7339 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:00.672413 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f577aee-73ca-4392-90fb-4da5a68d7339 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:00.672715 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f577aee-73ca-4392-90fb-4da5a68d7339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:00.684468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f577aee-73ca-4392-90fb-4da5a68d7339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:00.687603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f577aee-73ca-4392-90fb-4da5a68d7339 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:00.711659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1287/2583] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:02:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:02:01.081999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea2f0e-1d36-4596-a39b-20d6b0700460 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:01.084139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-23ea2f0e-1d36-4596-a39b-20d6b0700460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:01.100791 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea2f0e-1d36-4596-a39b-20d6b0700460 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:01.105831 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae961ae3-387e-4869-a68e-649077a7c9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:01.106727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae961ae3-387e-4869-a68e-649077a7c9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:01.107113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae961ae3-387e-4869-a68e-649077a7c9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:01.107276 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae961ae3-387e-4869-a68e-649077a7c9b8 admin admin] PATH_INFO: `/v3/projects/6559979e7e3b443a9e1633209bf113bb` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:01.108842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae961ae3-387e-4869-a68e-649077a7c9b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6559979e7e3b443a9e1633209bf113bb)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:01.110737 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae961ae3-387e-4869-a68e-649077a7c9b8 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:01.158401 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ae961ae3-387e-4869-a68e-649077a7c9b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6559979e7e3b443a9e1633209bf113bb', 'request_id': 'req-ae961ae3-387e-4869-a68e-649077a7c9b8'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:01.165300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ae961ae3-387e-4869-a68e-649077a7c9b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6559979e7e3b443a9e1633209bf113bb', 'request_id': 'req-ae961ae3-387e-4869-a68e-649077a7c9b8'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:01.171897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ae961ae3-387e-4869-a68e-649077a7c9b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6559979e7e3b443a9e1633209bf113bb', 'request_id': 'req-ae961ae3-387e-4869-a68e-649077a7c9b8'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:01.183002 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ae961ae3-387e-4869-a68e-649077a7c9b8 admin admin] The token cache is being invalidate because project 6559979e7e3b443a9e1633209bf113bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:02:01.183289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ae961ae3-387e-4869-a68e-649077a7c9b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:01.183961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-ae961ae3-387e-4869-a68e-649077a7c9b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:01.185946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1297/2584] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:02:01 2022] DELETE /identity/v3/projects/6559979e7e3b443a9e1633209bf113bb => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:02:01.190804 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2390ea6-8ec1-4ce3-a77d-ec05cfe0bc8a None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:01.191099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2390ea6-8ec1-4ce3-a77d-ec05cfe0bc8a None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:01.191378 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2390ea6-8ec1-4ce3-a77d-ec05cfe0bc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:01.209470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-c2390ea6-8ec1-4ce3-a77d-ec05cfe0bc8a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:02:01.210693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c2390ea6-8ec1-4ce3-a77d-ec05cfe0bc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:01.269358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1288/2585] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:02:01.334684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3663d-8147-4093-bed2-53c6d1211287 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:01.334966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3663d-8147-4093-bed2-53c6d1211287 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:01.335245 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3663d-8147-4093-bed2-53c6d1211287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:01.353682 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-a0e3663d-8147-4093-bed2-53c6d1211287 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:02:01.355405 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a0e3663d-8147-4093-bed2-53c6d1211287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:01.363540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1298/2586] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:02:01.368055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2390ea6-8ec1-4ce3-a77d-ec05cfe0bc8a None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:01.381130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2390ea6-8ec1-4ce3-a77d-ec05cfe0bc8a None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:01.386108 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5770046-1fde-4fe2-b0b1-4855a384708c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:01.386766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5770046-1fde-4fe2-b0b1-4855a384708c admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:01.387054 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5770046-1fde-4fe2-b0b1-4855a384708c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:01.387338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5770046-1fde-4fe2-b0b1-4855a384708c admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:01.388108 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5770046-1fde-4fe2-b0b1-4855a384708c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:02:01.388536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5770046-1fde-4fe2-b0b1-4855a384708c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:01.390157 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5770046-1fde-4fe2-b0b1-4855a384708c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:01.402077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1289/2587] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:02:01.406415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3663d-8147-4093-bed2-53c6d1211287 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:01.415636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3663d-8147-4093-bed2-53c6d1211287 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:01.422222 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fe33e-6ab7-4901-8bb4-bc9bf5656f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:01.422893 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fe33e-6ab7-4901-8bb4-bc9bf5656f66 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:01.423181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fe33e-6ab7-4901-8bb4-bc9bf5656f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:01.423460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fe33e-6ab7-4901-8bb4-bc9bf5656f66 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:01.424195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fe33e-6ab7-4901-8bb4-bc9bf5656f66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:01.426102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455fe33e-6ab7-4901-8bb4-bc9bf5656f66 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:01.453117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1299/2588] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:02:01 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:02:01.457243 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5770046-1fde-4fe2-b0b1-4855a384708c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:01.468742 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5770046-1fde-4fe2-b0b1-4855a384708c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:01.491401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:02:01.529219 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0940d-cf7f-4763-95a5-8a5338cf7c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:01.529818 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0940d-cf7f-4763-95a5-8a5338cf7c2e admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:01.530107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0940d-cf7f-4763-95a5-8a5338cf7c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:01.530386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc0940d-cf7f-4763-95a5-8a5338cf7c2e admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:01.531076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0940d-cf7f-4763-95a5-8a5338cf7c2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:01.533033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc0940d-cf7f-4763-95a5-8a5338cf7c2e admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:01.552433 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1290/2589] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:02:01 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:02:01.556664 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fe33e-6ab7-4901-8bb4-bc9bf5656f66 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:01.566783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:02:01.571989 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fe33e-6ab7-4901-8bb4-bc9bf5656f66 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:01.579025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce1640-92ba-49d5-94be-9e40c4e83a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:01.579992 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce1640-92ba-49d5-94be-9e40c4e83a7b admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:01.580498 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce1640-92ba-49d5-94be-9e40c4e83a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:01.580776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce1640-92ba-49d5-94be-9e40c4e83a7b admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:01.581541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce1640-92ba-49d5-94be-9e40c4e83a7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:01.583156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce1640-92ba-49d5-94be-9e40c4e83a7b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:01.595302 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1300/2590] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:02:01.599401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0940d-cf7f-4763-95a5-8a5338cf7c2e admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:01.610130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc0940d-cf7f-4763-95a5-8a5338cf7c2e admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:01.614745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117c6eb5-d9fb-4e14-b0f1-0ea25f17ce9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:01.615390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c6eb5-d9fb-4e14-b0f1-0ea25f17ce9b admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:01.615673 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c6eb5-d9fb-4e14-b0f1-0ea25f17ce9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:01.615972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117c6eb5-d9fb-4e14-b0f1-0ea25f17ce9b admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:01.616692 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c6eb5-d9fb-4e14-b0f1-0ea25f17ce9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:01.618274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117c6eb5-d9fb-4e14-b0f1-0ea25f17ce9b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:01.628396 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1291/2591] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:02:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:02:01.632463 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce1640-92ba-49d5-94be-9e40c4e83a7b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:01.642698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce1640-92ba-49d5-94be-9e40c4e83a7b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:01.647314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b67a66-5360-4776-91cd-56c6aef5fae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:01.648226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b67a66-5360-4776-91cd-56c6aef5fae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:01.648513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b67a66-5360-4776-91cd-56c6aef5fae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:01.648790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b67a66-5360-4776-91cd-56c6aef5fae4 admin admin] PATH_INFO: `/v3/projects/73ce085523d84754b8d107b5c0936491/users/d8dd1356e5bc4d35aca1bf1edf6f2140/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:01.664553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b67a66-5360-4776-91cd-56c6aef5fae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ce085523d84754b8d107b5c0936491, user_id=d8dd1356e5bc4d35aca1bf1edf6f2140, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:01.667892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b67a66-5360-4776-91cd-56c6aef5fae4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:01.682909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1301/2592] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:02:01 2022] PUT /identity/v3/projects/73ce085523d84754b8d107b5c0936491/users/d8dd1356e5bc4d35aca1bf1edf6f2140/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:02:01.688419 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184d4d0-bea5-4f6e-a3d4-841ceb5cc6cf None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:01.688714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184d4d0-bea5-4f6e-a3d4-841ceb5cc6cf None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:01.689004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184d4d0-bea5-4f6e-a3d4-841ceb5cc6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:01.707621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-8184d4d0-bea5-4f6e-a3d4-841ceb5cc6cf None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:02:01.708907 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-8184d4d0-bea5-4f6e-a3d4-841ceb5cc6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:01.769996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1292/2593] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:02:01.780672 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b67a66-5360-4776-91cd-56c6aef5fae4 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:01.782285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-84b67a66-5360-4776-91cd-56c6aef5fae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:01.850906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b67a66-5360-4776-91cd-56c6aef5fae4 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:01.855478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e4bc9-9879-4e99-b868-f4411d91ab93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:01.856177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e4bc9-9879-4e99-b868-f4411d91ab93 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:01.856468 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e4bc9-9879-4e99-b868-f4411d91ab93 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:01.856753 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e4bc9-9879-4e99-b868-f4411d91ab93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:01.871617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e4bc9-9879-4e99-b868-f4411d91ab93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:01.873121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e4bc9-9879-4e99-b868-f4411d91ab93 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:01.883491 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1302/2594] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:02:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:02:02.120128 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184d4d0-bea5-4f6e-a3d4-841ceb5cc6cf None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:02.121695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-8184d4d0-bea5-4f6e-a3d4-841ceb5cc6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:02.135907 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184d4d0-bea5-4f6e-a3d4-841ceb5cc6cf None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:02.140756 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae172a92-3246-4a4d-a162-e5bc8a5bc3d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:02.141447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae172a92-3246-4a4d-a162-e5bc8a5bc3d1 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:02.141813 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae172a92-3246-4a4d-a162-e5bc8a5bc3d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:02.142100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae172a92-3246-4a4d-a162-e5bc8a5bc3d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:02.144094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ae172a92-3246-4a4d-a162-e5bc8a5bc3d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:02.161168 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae172a92-3246-4a4d-a162-e5bc8a5bc3d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:02.162807 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae172a92-3246-4a4d-a162-e5bc8a5bc3d1 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:02.180283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1293/2595] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:02:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:02:02.635025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ad615-fc26-4977-85b7-3628d5135ee6 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:02.635578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ad615-fc26-4977-85b7-3628d5135ee6 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:02.635578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724ad615-fc26-4977-85b7-3628d5135ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:02.664795 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-724ad615-fc26-4977-85b7-3628d5135ee6 None None] MFA Rules not processed for user `d8dd1356e5bc4d35aca1bf1edf6f2140`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:02:02.666024 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-724ad615-fc26-4977-85b7-3628d5135ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:02.742994 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1303/2596] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:02:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:02:02.755866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae172a92-3246-4a4d-a162-e5bc8a5bc3d1 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:02.757661 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ae172a92-3246-4a4d-a162-e5bc8a5bc3d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:02.842150 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae172a92-3246-4a4d-a162-e5bc8a5bc3d1 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:02.848371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094a540-cd71-4291-8182-03d9a28699f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:02.849525 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094a540-cd71-4291-8182-03d9a28699f3 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:02.850057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094a540-cd71-4291-8182-03d9a28699f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:02.850542 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1094a540-cd71-4291-8182-03d9a28699f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:02.866513 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094a540-cd71-4291-8182-03d9a28699f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:02.869822 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1094a540-cd71-4291-8182-03d9a28699f3 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:02.887413 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1294/2597] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:02:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:02:05.382558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f422d7-9df5-4d1e-8b62-158d783d8c00 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:05.383662 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f422d7-9df5-4d1e-8b62-158d783d8c00 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:05.383749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f422d7-9df5-4d1e-8b62-158d783d8c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:05.404595 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-e2f422d7-9df5-4d1e-8b62-158d783d8c00 None None] MFA Rules not processed for user `d8dd1356e5bc4d35aca1bf1edf6f2140`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:02:05.405974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e2f422d7-9df5-4d1e-8b62-158d783d8c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:05.413675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1304/2598] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:02:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:02:06.156937 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094a540-cd71-4291-8182-03d9a28699f3 service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:06.173853 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1094a540-cd71-4291-8182-03d9a28699f3 service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:06.177819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fa4d0-e76c-448f-ba09-6a45c7d597d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:06.178534 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fa4d0-e76c-448f-ba09-6a45c7d597d3 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:06.178876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fa4d0-e76c-448f-ba09-6a45c7d597d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:06.179178 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fa4d0-e76c-448f-ba09-6a45c7d597d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:06.181150 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b15fa4d0-e76c-448f-ba09-6a45c7d597d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:06.236539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:02:06.322081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fa4d0-e76c-448f-ba09-6a45c7d597d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:06.325503 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fa4d0-e76c-448f-ba09-6a45c7d597d3 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:06.368854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1295/2599] 10.222.0.64 () {60 vars in 1396 bytes} [Thu Jul 14 18:02:06 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3252 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 18:02:09.514818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be8df1-fb9a-4158-a0b4-288a504c0f16 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:09.515655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be8df1-fb9a-4158-a0b4-288a504c0f16 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:09.515655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be8df1-fb9a-4158-a0b4-288a504c0f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:09.581978 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-03be8df1-fb9a-4158-a0b4-288a504c0f16 None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:02:09.583527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-03be8df1-fb9a-4158-a0b4-288a504c0f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:09.623495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:02:09.697502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1305/2600] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 18:02:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:02:09.773527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fa4d0-e76c-448f-ba09-6a45c7d597d3 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:09.775899 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b15fa4d0-e76c-448f-ba09-6a45c7d597d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:10.029451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fa4d0-e76c-448f-ba09-6a45c7d597d3 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:10.041839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ea6af-ee18-491d-ac96-dd25c898a380 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:10.043017 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea6af-ee18-491d-ac96-dd25c898a380 service swift] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:10.043527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea6af-ee18-491d-ac96-dd25c898a380 service swift] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:10.044225 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea6af-ee18-491d-ac96-dd25c898a380 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:10.065617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ea6af-ee18-491d-ac96-dd25c898a380 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:10.070138 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ea6af-ee18-491d-ac96-dd25c898a380 service swift] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:10.092539 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1296/2601] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 18:02:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 18:02:11.198528 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be8df1-fb9a-4158-a0b4-288a504c0f16 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:11.200153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-03be8df1-fb9a-4158-a0b4-288a504c0f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:11.239829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be8df1-fb9a-4158-a0b4-288a504c0f16 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:11.245084 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdb303-2a02-4bf3-a66b-a5d535e49c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:11.246452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb303-2a02-4bf3-a66b-a5d535e49c80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:11.247239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb303-2a02-4bf3-a66b-a5d535e49c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:11.247851 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bdb303-2a02-4bf3-a66b-a5d535e49c80 admin admin] PATH_INFO: `/v3/users/5d24ace76e3e4f10b63a53bb7482ac07` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:11.249677 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdb303-2a02-4bf3-a66b-a5d535e49c80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d24ace76e3e4f10b63a53bb7482ac07)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:11.251835 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bdb303-2a02-4bf3-a66b-a5d535e49c80 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:11.326107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-01bdb303-2a02-4bf3-a66b-a5d535e49c80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d24ace76e3e4f10b63a53bb7482ac07'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:11.345604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-01bdb303-2a02-4bf3-a66b-a5d535e49c80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d24ace76e3e4f10b63a53bb7482ac07'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:11.359757 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-01bdb303-2a02-4bf3-a66b-a5d535e49c80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d24ace76e3e4f10b63a53bb7482ac07'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:11.360437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-01bdb303-2a02-4bf3-a66b-a5d535e49c80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d24ace76e3e4f10b63a53bb7482ac07'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:11.370072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-01bdb303-2a02-4bf3-a66b-a5d535e49c80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d24ace76e3e4f10b63a53bb7482ac07'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:11.400252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1306/2602] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:02:11 2022] DELETE /identity/v3/users/5d24ace76e3e4f10b63a53bb7482ac07 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:02:11.736052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ea6af-ee18-491d-ac96-dd25c898a380 service swift] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:11.767913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ea6af-ee18-491d-ac96-dd25c898a380 service swift] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:11.932939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:11.934009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:11.934302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:11.934588 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] PATH_INFO: `/v3/projects/96004c35e9a84f2ab6b0601e28f45627` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:11.939545 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96004c35e9a84f2ab6b0601e28f45627)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:11.941691 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:11.976548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:02:12.034167 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96004c35e9a84f2ab6b0601e28f45627', 'request_id': 'req-b92211ca-414b-43a0-af88-ff9d439a8505'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:12.049112 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96004c35e9a84f2ab6b0601e28f45627', 'request_id': 'req-b92211ca-414b-43a0-af88-ff9d439a8505'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:12.062328 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96004c35e9a84f2ab6b0601e28f45627', 'request_id': 'req-b92211ca-414b-43a0-af88-ff9d439a8505'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:12.086066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] The token cache is being invalidate because project 96004c35e9a84f2ab6b0601e28f45627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:02:12.086785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:12.087776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:12.089696 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1297/2603] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:02:11 2022] DELETE /identity/v3/projects/96004c35e9a84f2ab6b0601e28f45627 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:02:12.100166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdb303-2a02-4bf3-a66b-a5d535e49c80 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:12.100166 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-01bdb303-2a02-4bf3-a66b-a5d535e49c80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:12.116392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bdb303-2a02-4bf3-a66b-a5d535e49c80 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:12.208381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d8460-b5fc-4633-b459-6b4d1deea056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:12.209955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d8460-b5fc-4633-b459-6b4d1deea056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:12.210448 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d8460-b5fc-4633-b459-6b4d1deea056 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:12.210943 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d8460-b5fc-4633-b459-6b4d1deea056 admin admin] PATH_INFO: `/v3/users/c4e1e194e6dc4a3dafbba25de07d6e43` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:12.215429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d8460-b5fc-4633-b459-6b4d1deea056 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4e1e194e6dc4a3dafbba25de07d6e43)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:12.217432 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d8460-b5fc-4633-b459-6b4d1deea056 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:12.326020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:02:12.339745 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-407d8460-b5fc-4633-b459-6b4d1deea056 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4e1e194e6dc4a3dafbba25de07d6e43'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:12.370152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-407d8460-b5fc-4633-b459-6b4d1deea056 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4e1e194e6dc4a3dafbba25de07d6e43'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:12.391037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-407d8460-b5fc-4633-b459-6b4d1deea056 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4e1e194e6dc4a3dafbba25de07d6e43'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:12.392982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-407d8460-b5fc-4633-b459-6b4d1deea056 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4e1e194e6dc4a3dafbba25de07d6e43'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:12.408108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-407d8460-b5fc-4633-b459-6b4d1deea056 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4e1e194e6dc4a3dafbba25de07d6e43'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:12.416401 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1307/2604] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:02:12 2022] DELETE /identity/v3/users/c4e1e194e6dc4a3dafbba25de07d6e43 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:02:12.779976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:12.781949 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:12.805439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92211ca-414b-43a0-af88-ff9d439a8505 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:12.892283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93453909-712f-4541-b51c-fc1d7f5ddcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:12.893084 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93453909-712f-4541-b51c-fc1d7f5ddcdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:12.893370 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93453909-712f-4541-b51c-fc1d7f5ddcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:12.893650 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93453909-712f-4541-b51c-fc1d7f5ddcdd admin admin] PATH_INFO: `/v3/projects/4108e5427f894fd39fc682d56b455b8f` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:12.895682 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93453909-712f-4541-b51c-fc1d7f5ddcdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4108e5427f894fd39fc682d56b455b8f)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:12.897766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93453909-712f-4541-b51c-fc1d7f5ddcdd admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:12.976524 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-93453909-712f-4541-b51c-fc1d7f5ddcdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4108e5427f894fd39fc682d56b455b8f', 'request_id': 'req-93453909-712f-4541-b51c-fc1d7f5ddcdd'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:12.987526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-93453909-712f-4541-b51c-fc1d7f5ddcdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4108e5427f894fd39fc682d56b455b8f', 'request_id': 'req-93453909-712f-4541-b51c-fc1d7f5ddcdd'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:12.996582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-93453909-712f-4541-b51c-fc1d7f5ddcdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4108e5427f894fd39fc682d56b455b8f', 'request_id': 'req-93453909-712f-4541-b51c-fc1d7f5ddcdd'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:13.017720 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-93453909-712f-4541-b51c-fc1d7f5ddcdd admin admin] The token cache is being invalidate because project 4108e5427f894fd39fc682d56b455b8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:02:13.018302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-93453909-712f-4541-b51c-fc1d7f5ddcdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:13.019063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-93453909-712f-4541-b51c-fc1d7f5ddcdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:02:13.020834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1298/2605] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:02:12 2022] DELETE /identity/v3/projects/4108e5427f894fd39fc682d56b455b8f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:02:17.020508 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d8460-b5fc-4633-b459-6b4d1deea056 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:17.022021 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-407d8460-b5fc-4633-b459-6b4d1deea056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:17.136986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d8460-b5fc-4633-b459-6b4d1deea056 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:17.222086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1de0b-62b4-4ea8-95bd-e37344d3d181 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:17.222758 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1de0b-62b4-4ea8-95bd-e37344d3d181 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:17.223043 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1de0b-62b4-4ea8-95bd-e37344d3d181 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:17.223327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1de0b-62b4-4ea8-95bd-e37344d3d181 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:17.225891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-92d1de0b-62b4-4ea8-95bd-e37344d3d181 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:17.241209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1de0b-62b4-4ea8-95bd-e37344d3d181 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:17.242891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1de0b-62b4-4ea8-95bd-e37344d3d181 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:17.254248 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1308/2606] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:02:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:02:23.011345 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=69,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:02:25.462819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73163742-7e50-40b3-acdd-d25859fa7db4 None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:25.463613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73163742-7e50-40b3-acdd-d25859fa7db4 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:25.463613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73163742-7e50-40b3-acdd-d25859fa7db4 None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:25.464795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1299/2607] 10.222.0.64 () {54 vars in 921 bytes} [Thu Jul 14 18:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:02:25.470642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da89e3-d546-43a8-8dfc-2a330375cd40 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:25.471100 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da89e3-d546-43a8-8dfc-2a330375cd40 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:25.471235 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39da89e3-d546-43a8-8dfc-2a330375cd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:25.511394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-39da89e3-d546-43a8-8dfc-2a330375cd40 None None] MFA Rules not processed for user `6423f54152064b909d03c0908d636993`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:02:25.513229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-39da89e3-d546-43a8-8dfc-2a330375cd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:25.573407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:02:25.591607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1309/2608] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Jul 14 18:02:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:02:25.605001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73163742-7e50-40b3-acdd-d25859fa7db4 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:25.607220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-73163742-7e50-40b3-acdd-d25859fa7db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:25.679953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73163742-7e50-40b3-acdd-d25859fa7db4 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:25.691211 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24351e02-a015-4cc4-8b68-ffdfb905a76c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2b4bf2aa1744a46b6d2a1b5a136a0e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:25.691994 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351e02-a015-4cc4-8b68-ffdfb905a76c service glance] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:25.692418 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351e02-a015-4cc4-8b68-ffdfb905a76c service glance] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:25.692708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351e02-a015-4cc4-8b68-ffdfb905a76c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:25.704527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24351e02-a015-4cc4-8b68-ffdfb905a76c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:25.705971 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24351e02-a015-4cc4-8b68-ffdfb905a76c service glance] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:25.718659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1300/2609] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:02:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 18:02:25.789991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175fb3b-263c-43b6-b0f6-d167d1f3bf8e None None] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:25.790309 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175fb3b-263c-43b6-b0f6-d167d1f3bf8e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:25.790589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175fb3b-263c-43b6-b0f6-d167d1f3bf8e None None] PATH_INFO: `/` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:25.792008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1310/2610] 10.222.0.64 () {54 vars in 921 bytes} [Thu Jul 14 18:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 18:02:25.797548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749bfd5-e0d6-4b7b-af9f-d0a21044bc49 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:25.797842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749bfd5-e0d6-4b7b-af9f-d0a21044bc49 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:25.798127 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749bfd5-e0d6-4b7b-af9f-d0a21044bc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:25.837741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-2749bfd5-e0d6-4b7b-af9f-d0a21044bc49 None None] MFA Rules not processed for user `a5adec9924444e29b8964f7969452592`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:02:25.838895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-2749bfd5-e0d6-4b7b-af9f-d0a21044bc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:25.845865 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1301/2611] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Jul 14 18:02:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:02:25.858240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175fb3b-263c-43b6-b0f6-d167d1f3bf8e None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:25.860041 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4175fb3b-263c-43b6-b0f6-d167d1f3bf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:25.874504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175fb3b-263c-43b6-b0f6-d167d1f3bf8e None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:25.877966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa6a45-5731-447a-8842-018fe615c6de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:25.878615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa6a45-5731-447a-8842-018fe615c6de service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:25.878897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa6a45-5731-447a-8842-018fe615c6de service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:25.879170 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa6a45-5731-447a-8842-018fe615c6de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:25.886452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa6a45-5731-447a-8842-018fe615c6de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:25.887864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa6a45-5731-447a-8842-018fe615c6de service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:25.895814 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1311/2612] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:02:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:02:25.982443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fed69-2bff-410e-8109-4c5ed68b236b None None] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:25.982828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fed69-2bff-410e-8109-4c5ed68b236b None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:25.983339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fed69-2bff-410e-8109-4c5ed68b236b None None] PATH_INFO: `/` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:25.984977 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1302/2613] 10.222.0.64 () {54 vars in 921 bytes} [Thu Jul 14 18:02:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 18:02:25.993808 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae4a43-ed5d-47d4-98c9-06866e1377d1 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:25.994108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae4a43-ed5d-47d4-98c9-06866e1377d1 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:25.994389 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae4a43-ed5d-47d4-98c9-06866e1377d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:26.012649 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-a0ae4a43-ed5d-47d4-98c9-06866e1377d1 None None] MFA Rules not processed for user `6423f54152064b909d03c0908d636993`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:02:26.013854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a0ae4a43-ed5d-47d4-98c9-06866e1377d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:26.023055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1312/2614] 10.222.0.64 () {58 vars in 1018 bytes} [Thu Jul 14 18:02:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:02:26.033885 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fed69-2bff-410e-8109-4c5ed68b236b None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:26.045762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fed69-2bff-410e-8109-4c5ed68b236b None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:26.049319 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfd4b3-7c21-4f6a-8230-4d0d7546b89f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:26.049953 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfd4b3-7c21-4f6a-8230-4d0d7546b89f service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:26.050237 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfd4b3-7c21-4f6a-8230-4d0d7546b89f service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:26.050518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfd4b3-7c21-4f6a-8230-4d0d7546b89f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:26.060614 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfd4b3-7c21-4f6a-8230-4d0d7546b89f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:26.062039 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfd4b3-7c21-4f6a-8230-4d0d7546b89f service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:26.073451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1303/2615] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:02:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:02:27.878484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336a298-2acf-43ad-a443-9340ca883e0a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:27.878968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336a298-2acf-43ad-a443-9340ca883e0a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:27.879073 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f336a298-2acf-43ad-a443-9340ca883e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:27.897674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f336a298-2acf-43ad-a443-9340ca883e0a None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:02:27.898889 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f336a298-2acf-43ad-a443-9340ca883e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:27.963175 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1313/2616] 10.222.0.64 () {58 vars in 1009 bytes} [Thu Jul 14 18:02:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:02:27.978687 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfd4b3-7c21-4f6a-8230-4d0d7546b89f service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:27.980190 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b8dfd4b3-7c21-4f6a-8230-4d0d7546b89f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:28.049238 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfd4b3-7c21-4f6a-8230-4d0d7546b89f service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:28.054158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ea2c8-1c2c-4b26-aa55-56a67b575a83 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:28.054812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ea2c8-1c2c-4b26-aa55-56a67b575a83 service swift] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:28.055096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ea2c8-1c2c-4b26-aa55-56a67b575a83 service swift] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:28.055371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ea2c8-1c2c-4b26-aa55-56a67b575a83 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:28.063760 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ea2c8-1c2c-4b26-aa55-56a67b575a83 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:28.065379 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ea2c8-1c2c-4b26-aa55-56a67b575a83 service swift] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:28.074379 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1304/2617] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 18:02:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 18:02:37.955391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=36 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:02:38.070727 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=73 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:02:53.077398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680bcb6-aee1-4f4f-bc0e-dba42bb9721a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:53.077912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680bcb6-aee1-4f4f-bc0e-dba42bb9721a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:53.078020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9680bcb6-aee1-4f4f-bc0e-dba42bb9721a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:53.098569 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-9680bcb6-aee1-4f4f-bc0e-dba42bb9721a None None] MFA Rules not processed for user `0cd1747822c3493e8aec150a12409fbd`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:02:53.099889 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9680bcb6-aee1-4f4f-bc0e-dba42bb9721a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:02:53.108380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1314/2618] 10.222.0.64 () {58 vars in 967 bytes} [Thu Jul 14 18:02:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:02:53.121081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ea2c8-1c2c-4b26-aa55-56a67b575a83 service swift] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:02:53.131672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ea2c8-1c2c-4b26-aa55-56a67b575a83 service swift] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:02:53.137120 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1839d9de-d26c-4a32-8935-6daea8a30b93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97f15306528e415baa5c69f12f22c75e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:02:53.137750 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839d9de-d26c-4a32-8935-6daea8a30b93 service swift] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:02:53.138034 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839d9de-d26c-4a32-8935-6daea8a30b93 service swift] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:02:53.138350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1839d9de-d26c-4a32-8935-6daea8a30b93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:02:53.146321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839d9de-d26c-4a32-8935-6daea8a30b93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:02:53.148572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1839d9de-d26c-4a32-8935-6daea8a30b93 service swift] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:02:53.156532 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1305/2619] 10.222.0.64 () {60 vars in 1384 bytes} [Thu Jul 14 18:02:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 18:03:02.116420 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680bcb6-aee1-4f4f-bc0e-dba42bb9721a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:03:02.117850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9680bcb6-aee1-4f4f-bc0e-dba42bb9721a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:02.133771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9680bcb6-aee1-4f4f-bc0e-dba42bb9721a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:03:02.139104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5804d-19eb-4836-93b7-df635e80fb56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:03:02.139744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5804d-19eb-4836-93b7-df635e80fb56 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:02.140063 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5804d-19eb-4836-93b7-df635e80fb56 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:02.140341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd5804d-19eb-4836-93b7-df635e80fb56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:02.142089 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3dd5804d-19eb-4836-93b7-df635e80fb56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:02.219699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5804d-19eb-4836-93b7-df635e80fb56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:03:02.221335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd5804d-19eb-4836-93b7-df635e80fb56 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:03:02.240347 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1315/2620] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:03:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:03:03.153343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=6 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:03:12.237666 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=44 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:03:35.371714 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30e1f2-592d-4e9d-8a0c-82ceec7d3025 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:35.372252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30e1f2-592d-4e9d-8a0c-82ceec7d3025 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:35.372252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30e1f2-592d-4e9d-8a0c-82ceec7d3025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:35.390090 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-2a30e1f2-592d-4e9d-8a0c-82ceec7d3025 None None] MFA Rules not processed for user `d8dd1356e5bc4d35aca1bf1edf6f2140`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:03:35.391274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-2a30e1f2-592d-4e9d-8a0c-82ceec7d3025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:35.452433 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1306/2621] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:03:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:03:35.677817 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5804d-19eb-4836-93b7-df635e80fb56 service nova] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:03:35.679717 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3dd5804d-19eb-4836-93b7-df635e80fb56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:35.691434 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd5804d-19eb-4836-93b7-df635e80fb56 service nova] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:03:35.695706 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8399b9b-31a4-4148-bcb0-4b6f4c6ffe1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:03:35.696391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8399b9b-31a4-4148-bcb0-4b6f4c6ffe1e service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:35.696675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8399b9b-31a4-4148-bcb0-4b6f4c6ffe1e service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:35.696955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8399b9b-31a4-4148-bcb0-4b6f4c6ffe1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:35.698667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e8399b9b-31a4-4148-bcb0-4b6f4c6ffe1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:35.708157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8399b9b-31a4-4148-bcb0-4b6f4c6ffe1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:03:35.709576 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8399b9b-31a4-4148-bcb0-4b6f4c6ffe1e service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:03:35.717610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1316/2622] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:03:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:03:36.047860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30e1f2-592d-4e9d-8a0c-82ceec7d3025 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:03:36.049434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-2a30e1f2-592d-4e9d-8a0c-82ceec7d3025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:36.135468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30e1f2-592d-4e9d-8a0c-82ceec7d3025 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:03:36.140913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252cc37-0979-4202-b7cf-f2f455115907 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:03:36.141515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252cc37-0979-4202-b7cf-f2f455115907 service placement] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:36.141799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252cc37-0979-4202-b7cf-f2f455115907 service placement] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:36.142078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252cc37-0979-4202-b7cf-f2f455115907 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:36.143980 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f252cc37-0979-4202-b7cf-f2f455115907 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:36.156397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252cc37-0979-4202-b7cf-f2f455115907 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:03:36.158345 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252cc37-0979-4202-b7cf-f2f455115907 service placement] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:03:36.171574 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1307/2623] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:03:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:03:45.714022 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=6 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:03:46.165208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=57 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:03:47.007725 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8399b9b-31a4-4148-bcb0-4b6f4c6ffe1e service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:03:47.009299 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e8399b9b-31a4-4148-bcb0-4b6f4c6ffe1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:47.029251 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8399b9b-31a4-4148-bcb0-4b6f4c6ffe1e service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:03:47.035383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b8c0c-3ffb-4b72-a1b5-e29d06955189 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:03:47.036789 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b8c0c-3ffb-4b72-a1b5-e29d06955189 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:47.037786 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b8c0c-3ffb-4b72-a1b5-e29d06955189 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:47.038303 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b8c0c-3ffb-4b72-a1b5-e29d06955189 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:47.041641 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c91b8c0c-3ffb-4b72-a1b5-e29d06955189 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:47.058167 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b8c0c-3ffb-4b72-a1b5-e29d06955189 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:03:47.059801 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b8c0c-3ffb-4b72-a1b5-e29d06955189 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:03:47.072776 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1317/2624] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:03:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:03:51.062956 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252cc37-0979-4202-b7cf-f2f455115907 service placement] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:03:51.073089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252cc37-0979-4202-b7cf-f2f455115907 service placement] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:03:51.077142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735a197-391b-404d-adba-94e9de338335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:03:51.077824 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735a197-391b-404d-adba-94e9de338335 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:51.078112 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735a197-391b-404d-adba-94e9de338335 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:51.078395 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0735a197-391b-404d-adba-94e9de338335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:51.080492 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0735a197-391b-404d-adba-94e9de338335 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:51.156516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735a197-391b-404d-adba-94e9de338335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:03:51.158059 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0735a197-391b-404d-adba-94e9de338335 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:03:51.169908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1308/2625] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:03:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:03:54.375317 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b8c0c-3ffb-4b72-a1b5-e29d06955189 service nova] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:03:54.376875 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c91b8c0c-3ffb-4b72-a1b5-e29d06955189 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:54.390653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b8c0c-3ffb-4b72-a1b5-e29d06955189 service nova] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:03:54.395853 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6628f08-0460-461d-ab02-2823bb944da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:03:54.396923 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6628f08-0460-461d-ab02-2823bb944da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:54.397202 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6628f08-0460-461d-ab02-2823bb944da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:54.397477 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6628f08-0460-461d-ab02-2823bb944da2 admin admin] PATH_INFO: `/v3/users/84c38f6ff5af4ba79c80fb581baacb0b` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:54.398665 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6628f08-0460-461d-ab02-2823bb944da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c38f6ff5af4ba79c80fb581baacb0b)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:03:54.400894 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6628f08-0460-461d-ab02-2823bb944da2 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:03:54.452521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b6628f08-0460-461d-ab02-2823bb944da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c38f6ff5af4ba79c80fb581baacb0b'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:03:54.465335 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b6628f08-0460-461d-ab02-2823bb944da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c38f6ff5af4ba79c80fb581baacb0b'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:03:54.474381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b6628f08-0460-461d-ab02-2823bb944da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c38f6ff5af4ba79c80fb581baacb0b'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:03:54.475009 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b6628f08-0460-461d-ab02-2823bb944da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c38f6ff5af4ba79c80fb581baacb0b'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:03:54.482326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b6628f08-0460-461d-ab02-2823bb944da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c38f6ff5af4ba79c80fb581baacb0b'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:03:54.488551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1318/2626] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:03:54 2022] DELETE /identity/v3/users/84c38f6ff5af4ba79c80fb581baacb0b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:03:54.670192 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735a197-391b-404d-adba-94e9de338335 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:03:54.683407 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0735a197-391b-404d-adba-94e9de338335 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:03:54.748280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:03:54.749148 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:54.749427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:54.749698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26 admin admin] PATH_INFO: `/v3/projects/e51408d5653d460f82009b65e238281f` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:54.750867 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e51408d5653d460f82009b65e238281f)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:03:54.753452 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:03:54.800646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e51408d5653d460f82009b65e238281f', 'request_id': 'req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:03:54.807731 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e51408d5653d460f82009b65e238281f', 'request_id': 'req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:03:54.813312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e51408d5653d460f82009b65e238281f', 'request_id': 'req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:03:54.824774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26 admin admin] The token cache is being invalidate because project e51408d5653d460f82009b65e238281f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:03:54.825059 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:03:54.825803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-859aef7f-a9ae-4a97-bdd7-68b49dfe7c26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:03:54.827622 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1309/2627] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:03:54 2022] DELETE /identity/v3/projects/e51408d5653d460f82009b65e238281f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:03:54.833444 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d941a-8c1b-47e4-b772-870ff696d5bf None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:54.833762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d941a-8c1b-47e4-b772-870ff696d5bf None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:54.834050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d941a-8c1b-47e4-b772-870ff696d5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:54.851972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-2c1d941a-8c1b-47e4-b772-870ff696d5bf None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:03:54.853192 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-2c1d941a-8c1b-47e4-b772-870ff696d5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:54.913266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1319/2628] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:03:54.920877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b4b1b-b50e-47ae-af28-c8dbac88f573 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:54.921204 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b4b1b-b50e-47ae-af28-c8dbac88f573 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:54.921493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6b4b1b-b50e-47ae-af28-c8dbac88f573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:54.942410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5b6b4b1b-b50e-47ae-af28-c8dbac88f573 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:03:54.945031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5b6b4b1b-b50e-47ae-af28-c8dbac88f573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:54.953409 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1310/2629] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:03:54.958540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d941a-8c1b-47e4-b772-870ff696d5bf None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:03:54.968946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d941a-8c1b-47e4-b772-870ff696d5bf None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:03:54.973979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3485f3-e940-4a00-b06a-5f756c83f9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:03:54.974708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3485f3-e940-4a00-b06a-5f756c83f9ec admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:54.974953 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3485f3-e940-4a00-b06a-5f756c83f9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:54.975230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3485f3-e940-4a00-b06a-5f756c83f9ec admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:54.976010 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3485f3-e940-4a00-b06a-5f756c83f9ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:03:54.976330 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3485f3-e940-4a00-b06a-5f756c83f9ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:03:54.977955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3485f3-e940-4a00-b06a-5f756c83f9ec admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:03:54.991657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1320/2630] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:03:54.996455 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b4b1b-b50e-47ae-af28-c8dbac88f573 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:03:55.006754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6b4b1b-b50e-47ae-af28-c8dbac88f573 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:03:55.011470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb28d5e-388b-47f6-8888-a8d4198ece00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:03:55.012177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb28d5e-388b-47f6-8888-a8d4198ece00 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:55.012460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb28d5e-388b-47f6-8888-a8d4198ece00 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:55.012732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb28d5e-388b-47f6-8888-a8d4198ece00 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:55.013440 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb28d5e-388b-47f6-8888-a8d4198ece00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:03:55.015366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb28d5e-388b-47f6-8888-a8d4198ece00 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:03:55.030689 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:03:55.046006 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1311/2631] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:03:54 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:03:55.050743 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3485f3-e940-4a00-b06a-5f756c83f9ec admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:03:55.060884 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3485f3-e940-4a00-b06a-5f756c83f9ec admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:03:55.115429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:03:55.135042 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6591c4d-7d03-44af-8ab2-87f793aa0d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:03:55.135763 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6591c4d-7d03-44af-8ab2-87f793aa0d70 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:55.136091 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6591c4d-7d03-44af-8ab2-87f793aa0d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:55.136368 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6591c4d-7d03-44af-8ab2-87f793aa0d70 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:55.137090 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6591c4d-7d03-44af-8ab2-87f793aa0d70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:03:55.139189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6591c4d-7d03-44af-8ab2-87f793aa0d70 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:03:55.161669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1321/2632] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:03:55 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:03:55.166101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb28d5e-388b-47f6-8888-a8d4198ece00 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:03:55.177404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb28d5e-388b-47f6-8888-a8d4198ece00 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:03:55.182074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e74e9d-1e90-4d1a-88cb-f4ca72bed729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:03:55.182748 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e74e9d-1e90-4d1a-88cb-f4ca72bed729 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:55.183035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e74e9d-1e90-4d1a-88cb-f4ca72bed729 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:55.183314 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e74e9d-1e90-4d1a-88cb-f4ca72bed729 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:55.184085 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e74e9d-1e90-4d1a-88cb-f4ca72bed729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:03:55.185685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e74e9d-1e90-4d1a-88cb-f4ca72bed729 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:03:55.197264 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1312/2633] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:03:55.201551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6591c4d-7d03-44af-8ab2-87f793aa0d70 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:03:55.211755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6591c4d-7d03-44af-8ab2-87f793aa0d70 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:03:55.216930 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e7c94-e584-4016-9539-57e790dace61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:03:55.217640 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e7c94-e584-4016-9539-57e790dace61 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:55.217926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e7c94-e584-4016-9539-57e790dace61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:55.218200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e7c94-e584-4016-9539-57e790dace61 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:55.218960 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e7c94-e584-4016-9539-57e790dace61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:03:55.220593 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e7c94-e584-4016-9539-57e790dace61 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:03:55.232602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1322/2634] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:03:55.237378 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e74e9d-1e90-4d1a-88cb-f4ca72bed729 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:03:55.248301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e74e9d-1e90-4d1a-88cb-f4ca72bed729 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:03:55.253425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bab062-85cb-44b0-a9ea-15453fc19917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:03:55.254306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bab062-85cb-44b0-a9ea-15453fc19917 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:55.254589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bab062-85cb-44b0-a9ea-15453fc19917 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:55.254866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bab062-85cb-44b0-a9ea-15453fc19917 admin admin] PATH_INFO: `/v3/projects/cd7b0d70fffe4d37a84bd4ee73d044ca/users/d1a7b2376e83489b87feb4b9fd449d55/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:55.269162 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bab062-85cb-44b0-a9ea-15453fc19917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd7b0d70fffe4d37a84bd4ee73d044ca, user_id=d1a7b2376e83489b87feb4b9fd449d55, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:03:55.271769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bab062-85cb-44b0-a9ea-15453fc19917 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:03:55.288560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1313/2635] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:03:55 2022] PUT /identity/v3/projects/cd7b0d70fffe4d37a84bd4ee73d044ca/users/d1a7b2376e83489b87feb4b9fd449d55/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:03:55.294151 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf81f3-de7a-4682-800d-d5dc9295f988 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:55.294607 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf81f3-de7a-4682-800d-d5dc9295f988 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:55.294897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf81f3-de7a-4682-800d-d5dc9295f988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:55.321288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-65cf81f3-de7a-4682-800d-d5dc9295f988 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:03:55.322502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-65cf81f3-de7a-4682-800d-d5dc9295f988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:55.382856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1323/2636] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:03:55.392637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bab062-85cb-44b0-a9ea-15453fc19917 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:03:55.394536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a8bab062-85cb-44b0-a9ea-15453fc19917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:03:55.459602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bab062-85cb-44b0-a9ea-15453fc19917 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:03:55.463580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc118967-8023-4d88-8296-5320e8a6a8f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:03:55.464287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc118967-8023-4d88-8296-5320e8a6a8f7 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:03:55.464583 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc118967-8023-4d88-8296-5320e8a6a8f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:03:55.464863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc118967-8023-4d88-8296-5320e8a6a8f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:03:55.474731 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc118967-8023-4d88-8296-5320e8a6a8f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:03:55.476203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc118967-8023-4d88-8296-5320e8a6a8f7 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:03:55.485745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1314/2637] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:03:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:04:00.038286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb6bd7-e43c-4eef-a8c3-1a879c232939 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:00.038833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb6bd7-e43c-4eef-a8c3-1a879c232939 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:00.038999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb6bd7-e43c-4eef-a8c3-1a879c232939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:00.070956 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-6edb6bd7-e43c-4eef-a8c3-1a879c232939 None None] MFA Rules not processed for user `d1a7b2376e83489b87feb4b9fd449d55`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:04:00.072463 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-6edb6bd7-e43c-4eef-a8c3-1a879c232939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:00.149599 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1324/2638] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:04:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:04:00.164850 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc118967-8023-4d88-8296-5320e8a6a8f7 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:00.166320 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-bc118967-8023-4d88-8296-5320e8a6a8f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:00.243886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc118967-8023-4d88-8296-5320e8a6a8f7 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:00.249514 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e1307-a7f6-44ce-9846-e021ada467cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:00.250177 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e1307-a7f6-44ce-9846-e021ada467cf service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:00.250491 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e1307-a7f6-44ce-9846-e021ada467cf service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:00.251289 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e1307-a7f6-44ce-9846-e021ada467cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:00.260203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e1307-a7f6-44ce-9846-e021ada467cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:00.261821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406e1307-a7f6-44ce-9846-e021ada467cf service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:00.271363 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1315/2639] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:04:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:04:01.729902 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bab1f-d2b0-4baf-9829-58f5ed978808 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:01.730809 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bab1f-d2b0-4baf-9829-58f5ed978808 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:01.731288 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bab1f-d2b0-4baf-9829-58f5ed978808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:01.754378 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-464bab1f-d2b0-4baf-9829-58f5ed978808 None None] MFA Rules not processed for user `d1a7b2376e83489b87feb4b9fd449d55`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:04:01.755893 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-464bab1f-d2b0-4baf-9829-58f5ed978808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:01.763633 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1325/2640] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:04:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:04:10.267103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:04:11.747306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=75,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:04:40.238642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e1307-a7f6-44ce-9846-e021ada467cf service nova] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:40.264658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406e1307-a7f6-44ce-9846-e021ada467cf service nova] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:40.272536 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95299b-8d7f-4ad5-b933-6997cc54ab61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:40.273421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95299b-8d7f-4ad5-b933-6997cc54ab61 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:40.273823 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95299b-8d7f-4ad5-b933-6997cc54ab61 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:40.274193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f95299b-8d7f-4ad5-b933-6997cc54ab61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:40.277253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7f95299b-8d7f-4ad5-b933-6997cc54ab61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:40.299089 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95299b-8d7f-4ad5-b933-6997cc54ab61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:40.300963 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f95299b-8d7f-4ad5-b933-6997cc54ab61 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:40.320319 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1316/2641] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:04:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:04:45.079861 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bab1f-d2b0-4baf-9829-58f5ed978808 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:45.091546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bab1f-d2b0-4baf-9829-58f5ed978808 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:45.097596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dce4fb-e9b3-4fc4-b243-b55bbf6dccf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:45.098747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce4fb-e9b3-4fc4-b243-b55bbf6dccf8 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:45.099647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce4fb-e9b3-4fc4-b243-b55bbf6dccf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:45.099647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dce4fb-e9b3-4fc4-b243-b55bbf6dccf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:45.102532 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-59dce4fb-e9b3-4fc4-b243-b55bbf6dccf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:45.121038 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dce4fb-e9b3-4fc4-b243-b55bbf6dccf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:45.123719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dce4fb-e9b3-4fc4-b243-b55bbf6dccf8 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:45.141007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1326/2642] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:04:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:04:45.505585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95299b-8d7f-4ad5-b933-6997cc54ab61 service nova] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:45.507191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7f95299b-8d7f-4ad5-b933-6997cc54ab61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:45.522861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f95299b-8d7f-4ad5-b933-6997cc54ab61 service nova] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:45.532946 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949f50f-3463-4d22-863c-31e8c177f75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:45.533866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949f50f-3463-4d22-863c-31e8c177f75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:45.534173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949f50f-3463-4d22-863c-31e8c177f75a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:45.534443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0949f50f-3463-4d22-863c-31e8c177f75a admin admin] PATH_INFO: `/v3/users/d8dd1356e5bc4d35aca1bf1edf6f2140` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:45.535766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949f50f-3463-4d22-863c-31e8c177f75a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8dd1356e5bc4d35aca1bf1edf6f2140)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:45.537791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0949f50f-3463-4d22-863c-31e8c177f75a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:45.617810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0949f50f-3463-4d22-863c-31e8c177f75a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8dd1356e5bc4d35aca1bf1edf6f2140'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:45.636649 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0949f50f-3463-4d22-863c-31e8c177f75a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8dd1356e5bc4d35aca1bf1edf6f2140'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:45.638207 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0949f50f-3463-4d22-863c-31e8c177f75a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8dd1356e5bc4d35aca1bf1edf6f2140'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:45.651637 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0949f50f-3463-4d22-863c-31e8c177f75a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8dd1356e5bc4d35aca1bf1edf6f2140'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:45.652604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0949f50f-3463-4d22-863c-31e8c177f75a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8dd1356e5bc4d35aca1bf1edf6f2140'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:45.676220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1317/2643] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:04:45 2022] DELETE /identity/v3/users/d8dd1356e5bc4d35aca1bf1edf6f2140 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:04:45.929221 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dce4fb-e9b3-4fc4-b243-b55bbf6dccf8 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:45.962311 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dce4fb-e9b3-4fc4-b243-b55bbf6dccf8 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:46.085642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:46.087149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:46.087579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:46.087869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] PATH_INFO: `/v3/projects/73ce085523d84754b8d107b5c0936491` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:46.089380 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=73ce085523d84754b8d107b5c0936491)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:46.092749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:46.167136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73ce085523d84754b8d107b5c0936491', 'request_id': 'req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:46.178298 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73ce085523d84754b8d107b5c0936491', 'request_id': 'req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:46.187056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73ce085523d84754b8d107b5c0936491', 'request_id': 'req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:46.200897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] The token cache is being invalidate because project 73ce085523d84754b8d107b5c0936491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:04:46.201464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:46.204106 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:46.206110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1327/2644] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:04:45 2022] DELETE /identity/v3/projects/73ce085523d84754b8d107b5c0936491 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:04:48.812365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949f50f-3463-4d22-863c-31e8c177f75a admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:48.814442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0949f50f-3463-4d22-863c-31e8c177f75a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:48.831893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0949f50f-3463-4d22-863c-31e8c177f75a admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:48.891119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93987f7-80c5-42ce-af0f-b51dd1b0fc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:48.892152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93987f7-80c5-42ce-af0f-b51dd1b0fc65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:48.892432 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93987f7-80c5-42ce-af0f-b51dd1b0fc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:48.892712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93987f7-80c5-42ce-af0f-b51dd1b0fc65 admin admin] PATH_INFO: `/v3/users/c4178d34dc114f46af725c9dc362d224` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:48.893967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93987f7-80c5-42ce-af0f-b51dd1b0fc65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4178d34dc114f46af725c9dc362d224)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:48.895978 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93987f7-80c5-42ce-af0f-b51dd1b0fc65 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:48.945502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f93987f7-80c5-42ce-af0f-b51dd1b0fc65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4178d34dc114f46af725c9dc362d224'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:48.956230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:04:48.958604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f93987f7-80c5-42ce-af0f-b51dd1b0fc65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4178d34dc114f46af725c9dc362d224'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:48.959664 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f93987f7-80c5-42ce-af0f-b51dd1b0fc65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4178d34dc114f46af725c9dc362d224'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:48.968070 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f93987f7-80c5-42ce-af0f-b51dd1b0fc65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4178d34dc114f46af725c9dc362d224'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:48.968713 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f93987f7-80c5-42ce-af0f-b51dd1b0fc65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4178d34dc114f46af725c9dc362d224'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:48.981071 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1318/2645] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:04:48 2022] DELETE /identity/v3/users/c4178d34dc114f46af725c9dc362d224 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:04:49.167703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:49.169357 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:49.184378 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393e2f3-5858-4b37-8ac9-24d4c7c08d4a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:49.247218 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa284a9-b700-429d-b016-adcbc1f23293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:49.248171 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa284a9-b700-429d-b016-adcbc1f23293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:49.248460 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa284a9-b700-429d-b016-adcbc1f23293 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:49.248744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa284a9-b700-429d-b016-adcbc1f23293 admin admin] PATH_INFO: `/v3/projects/69d2b6a5a4534cadb34e626ce33238df` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:49.250039 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa284a9-b700-429d-b016-adcbc1f23293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d2b6a5a4534cadb34e626ce33238df)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:49.252054 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa284a9-b700-429d-b016-adcbc1f23293 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:49.275310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:04:49.306008 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7fa284a9-b700-429d-b016-adcbc1f23293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d2b6a5a4534cadb34e626ce33238df', 'request_id': 'req-7fa284a9-b700-429d-b016-adcbc1f23293'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:49.316109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7fa284a9-b700-429d-b016-adcbc1f23293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d2b6a5a4534cadb34e626ce33238df', 'request_id': 'req-7fa284a9-b700-429d-b016-adcbc1f23293'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:49.323570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7fa284a9-b700-429d-b016-adcbc1f23293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d2b6a5a4534cadb34e626ce33238df', 'request_id': 'req-7fa284a9-b700-429d-b016-adcbc1f23293'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:49.335213 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7fa284a9-b700-429d-b016-adcbc1f23293 admin admin] The token cache is being invalidate because project 69d2b6a5a4534cadb34e626ce33238df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:04:49.335511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7fa284a9-b700-429d-b016-adcbc1f23293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:49.336223 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7fa284a9-b700-429d-b016-adcbc1f23293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:49.338001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1328/2646] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:04:49 2022] DELETE /identity/v3/projects/69d2b6a5a4534cadb34e626ce33238df => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:04:49.342795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c824bce-a348-46f3-b4bb-cb1a49ceff6c None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:49.343166 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c824bce-a348-46f3-b4bb-cb1a49ceff6c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:49.343514 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c824bce-a348-46f3-b4bb-cb1a49ceff6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:49.364913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-6c824bce-a348-46f3-b4bb-cb1a49ceff6c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:04:49.366066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-6c824bce-a348-46f3-b4bb-cb1a49ceff6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:49.454860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1319/2647] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:04:49.462587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f15a46-f4be-42be-a3fa-50ca127bb44c None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:49.462875 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f15a46-f4be-42be-a3fa-50ca127bb44c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:49.463144 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f15a46-f4be-42be-a3fa-50ca127bb44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:49.482925 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-a7f15a46-f4be-42be-a3fa-50ca127bb44c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:04:49.484193 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a7f15a46-f4be-42be-a3fa-50ca127bb44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:49.493072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1329/2648] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:04:49.498372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c824bce-a348-46f3-b4bb-cb1a49ceff6c None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:49.508390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c824bce-a348-46f3-b4bb-cb1a49ceff6c None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:49.513348 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a1085-a97c-4113-ba08-aff0de005bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:49.514034 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a1085-a97c-4113-ba08-aff0de005bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:49.514325 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a1085-a97c-4113-ba08-aff0de005bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:49.514606 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a1085-a97c-4113-ba08-aff0de005bb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:49.515388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a1085-a97c-4113-ba08-aff0de005bb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:04:49.515701 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a1085-a97c-4113-ba08-aff0de005bb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:49.517436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a1085-a97c-4113-ba08-aff0de005bb2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:49.530069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1320/2649] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:04:49.534724 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f15a46-f4be-42be-a3fa-50ca127bb44c None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:49.545169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f15a46-f4be-42be-a3fa-50ca127bb44c None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:49.550065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ba372-dd42-4965-a1ac-06b8b28e8284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:49.550746 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ba372-dd42-4965-a1ac-06b8b28e8284 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:49.551029 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ba372-dd42-4965-a1ac-06b8b28e8284 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:49.551305 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ba372-dd42-4965-a1ac-06b8b28e8284 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:49.552050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ba372-dd42-4965-a1ac-06b8b28e8284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:49.553999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ba372-dd42-4965-a1ac-06b8b28e8284 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:49.582971 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1330/2650] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:04:49 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:04:49.587353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a1085-a97c-4113-ba08-aff0de005bb2 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:49.597511 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a1085-a97c-4113-ba08-aff0de005bb2 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:49.654650 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac264cca-fff8-4bfb-94fb-63ab78c81b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:49.655329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac264cca-fff8-4bfb-94fb-63ab78c81b25 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:49.655612 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac264cca-fff8-4bfb-94fb-63ab78c81b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:49.655890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac264cca-fff8-4bfb-94fb-63ab78c81b25 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:49.656621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac264cca-fff8-4bfb-94fb-63ab78c81b25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:49.658540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac264cca-fff8-4bfb-94fb-63ab78c81b25 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:49.683360 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1321/2651] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:04:49 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:04:49.687658 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ba372-dd42-4965-a1ac-06b8b28e8284 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:49.699945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ba372-dd42-4965-a1ac-06b8b28e8284 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:49.705479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dfa7d-b2d9-4691-bf8b-c3f0e7fbe09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:49.706120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dfa7d-b2d9-4691-bf8b-c3f0e7fbe09c admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:49.706405 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dfa7d-b2d9-4691-bf8b-c3f0e7fbe09c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:49.706688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03dfa7d-b2d9-4691-bf8b-c3f0e7fbe09c admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:49.707426 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03dfa7d-b2d9-4691-bf8b-c3f0e7fbe09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:49.709105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03dfa7d-b2d9-4691-bf8b-c3f0e7fbe09c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:49.721848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1331/2652] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 18:04:49.726964 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac264cca-fff8-4bfb-94fb-63ab78c81b25 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:49.739041 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac264cca-fff8-4bfb-94fb-63ab78c81b25 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:49.745702 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bae7c51-8ed8-4a25-921b-5283dcc9b65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:49.747041 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae7c51-8ed8-4a25-921b-5283dcc9b65c admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:49.747559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae7c51-8ed8-4a25-921b-5283dcc9b65c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:49.748862 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bae7c51-8ed8-4a25-921b-5283dcc9b65c admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:49.750365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae7c51-8ed8-4a25-921b-5283dcc9b65c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:49.753404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bae7c51-8ed8-4a25-921b-5283dcc9b65c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:49.767829 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1322/2653] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:04:49.772293 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dfa7d-b2d9-4691-bf8b-c3f0e7fbe09c admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:49.784625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03dfa7d-b2d9-4691-bf8b-c3f0e7fbe09c admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:49.790019 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422485c5-c340-46cf-87ff-dbe39653699e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:49.790935 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422485c5-c340-46cf-87ff-dbe39653699e admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:49.791214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422485c5-c340-46cf-87ff-dbe39653699e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:49.791494 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422485c5-c340-46cf-87ff-dbe39653699e admin admin] PATH_INFO: `/v3/projects/94b41787eebf4dc68871f38ff4ba12a8/users/c638a5ecc911478eaeb64aedd8d4c656/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:49.807770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422485c5-c340-46cf-87ff-dbe39653699e admin admin] RBAC: Authorizing `identity:create_grant(project_id=94b41787eebf4dc68871f38ff4ba12a8, user_id=c638a5ecc911478eaeb64aedd8d4c656, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:49.810800 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422485c5-c340-46cf-87ff-dbe39653699e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:49.826068 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1332/2654] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:04:49 2022] PUT /identity/v3/projects/94b41787eebf4dc68871f38ff4ba12a8/users/c638a5ecc911478eaeb64aedd8d4c656/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:04:49.838474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d07194-72c7-45c2-b1ce-eed9d8453d32 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:49.839377 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d07194-72c7-45c2-b1ce-eed9d8453d32 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:49.839766 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d07194-72c7-45c2-b1ce-eed9d8453d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:49.876145 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-c8d07194-72c7-45c2-b1ce-eed9d8453d32 None None] MFA Rules not processed for user `c638a5ecc911478eaeb64aedd8d4c656`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:04:49.877404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c8d07194-72c7-45c2-b1ce-eed9d8453d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:49.915765 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:04:49.954200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1323/2655] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:04:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:04:49.966735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422485c5-c340-46cf-87ff-dbe39653699e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:49.968612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-422485c5-c340-46cf-87ff-dbe39653699e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:50.050239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422485c5-c340-46cf-87ff-dbe39653699e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:50.056668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668fd9c5-0147-4e74-a62e-71a755943a88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:50.057341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668fd9c5-0147-4e74-a62e-71a755943a88 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:50.057628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668fd9c5-0147-4e74-a62e-71a755943a88 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:50.057904 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668fd9c5-0147-4e74-a62e-71a755943a88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:50.066718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668fd9c5-0147-4e74-a62e-71a755943a88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:50.068627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668fd9c5-0147-4e74-a62e-71a755943a88 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:50.079816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1333/2656] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:04:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:04:50.326739 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d07194-72c7-45c2-b1ce-eed9d8453d32 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:50.392889 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d07194-72c7-45c2-b1ce-eed9d8453d32 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:50.397490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455073ec-1ef7-4f69-9da1-9b2394ee1673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:50.398910 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455073ec-1ef7-4f69-9da1-9b2394ee1673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:50.399194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455073ec-1ef7-4f69-9da1-9b2394ee1673 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:50.399471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455073ec-1ef7-4f69-9da1-9b2394ee1673 admin admin] PATH_INFO: `/v3/users/c638a5ecc911478eaeb64aedd8d4c656` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:50.400712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455073ec-1ef7-4f69-9da1-9b2394ee1673 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c638a5ecc911478eaeb64aedd8d4c656)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:50.400712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455073ec-1ef7-4f69-9da1-9b2394ee1673 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:50.456818 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-455073ec-1ef7-4f69-9da1-9b2394ee1673 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c638a5ecc911478eaeb64aedd8d4c656'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:50.469075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-455073ec-1ef7-4f69-9da1-9b2394ee1673 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c638a5ecc911478eaeb64aedd8d4c656'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:50.469812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-455073ec-1ef7-4f69-9da1-9b2394ee1673 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c638a5ecc911478eaeb64aedd8d4c656'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:50.477411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-455073ec-1ef7-4f69-9da1-9b2394ee1673 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c638a5ecc911478eaeb64aedd8d4c656'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:50.477920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-455073ec-1ef7-4f69-9da1-9b2394ee1673 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c638a5ecc911478eaeb64aedd8d4c656'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:50.492561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1324/2657] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:04:50 2022] DELETE /identity/v3/users/c638a5ecc911478eaeb64aedd8d4c656 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:04:50.498565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeadad7-54de-4fb8-b06b-10507492932c None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:50.499101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeadad7-54de-4fb8-b06b-10507492932c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:50.499565 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeadad7-54de-4fb8-b06b-10507492932c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:50.528412 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-beeadad7-54de-4fb8-b06b-10507492932c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:04:50.530612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-beeadad7-54de-4fb8-b06b-10507492932c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:50.570500 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:04:50.606719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1334/2658] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:04:50.617352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455073ec-1ef7-4f69-9da1-9b2394ee1673 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:50.618851 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-455073ec-1ef7-4f69-9da1-9b2394ee1673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:50.677103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455073ec-1ef7-4f69-9da1-9b2394ee1673 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:50.681280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c511bf4-22e8-4e48-9ec7-a523e56ee4a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:50.681933 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c511bf4-22e8-4e48-9ec7-a523e56ee4a3 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:50.682208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c511bf4-22e8-4e48-9ec7-a523e56ee4a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:50.682481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c511bf4-22e8-4e48-9ec7-a523e56ee4a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:50.689642 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:04:50.692444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c511bf4-22e8-4e48-9ec7-a523e56ee4a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:50.695548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c511bf4-22e8-4e48-9ec7-a523e56ee4a3 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:50.707616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1325/2659] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:04:51.002474 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeadad7-54de-4fb8-b06b-10507492932c None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:51.004885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-beeadad7-54de-4fb8-b06b-10507492932c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:51.022056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeadad7-54de-4fb8-b06b-10507492932c None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:51.027968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d23e6-4a41-4001-910a-3b27a38b15e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:51.028888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d23e6-4a41-4001-910a-3b27a38b15e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:51.029180 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d23e6-4a41-4001-910a-3b27a38b15e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:51.029463 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d23e6-4a41-4001-910a-3b27a38b15e0 admin admin] PATH_INFO: `/v3/projects/94b41787eebf4dc68871f38ff4ba12a8` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:51.031035 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d23e6-4a41-4001-910a-3b27a38b15e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94b41787eebf4dc68871f38ff4ba12a8)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:51.033612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d23e6-4a41-4001-910a-3b27a38b15e0 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:51.103436 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-084d23e6-4a41-4001-910a-3b27a38b15e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94b41787eebf4dc68871f38ff4ba12a8', 'request_id': 'req-084d23e6-4a41-4001-910a-3b27a38b15e0'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:51.112687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-084d23e6-4a41-4001-910a-3b27a38b15e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94b41787eebf4dc68871f38ff4ba12a8', 'request_id': 'req-084d23e6-4a41-4001-910a-3b27a38b15e0'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:51.120975 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-084d23e6-4a41-4001-910a-3b27a38b15e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94b41787eebf4dc68871f38ff4ba12a8', 'request_id': 'req-084d23e6-4a41-4001-910a-3b27a38b15e0'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:51.134704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-084d23e6-4a41-4001-910a-3b27a38b15e0 admin admin] The token cache is being invalidate because project 94b41787eebf4dc68871f38ff4ba12a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:04:51.135156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-084d23e6-4a41-4001-910a-3b27a38b15e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:51.136217 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-084d23e6-4a41-4001-910a-3b27a38b15e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:04:51.138581 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1335/2660] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:04:51 2022] DELETE /identity/v3/projects/94b41787eebf4dc68871f38ff4ba12a8 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:04:51.143832 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367af601-b8a4-4d47-bcae-b0c689fcf9f4 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:51.144425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367af601-b8a4-4d47-bcae-b0c689fcf9f4 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:51.144636 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367af601-b8a4-4d47-bcae-b0c689fcf9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:51.163747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-367af601-b8a4-4d47-bcae-b0c689fcf9f4 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:04:51.165018 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-367af601-b8a4-4d47-bcae-b0c689fcf9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:51.230418 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1326/2661] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:04:51.238589 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8b82a-6033-412c-892f-d66c1d82148f None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:51.238927 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8b82a-6033-412c-892f-d66c1d82148f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:51.239150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8b82a-6033-412c-892f-d66c1d82148f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:51.258509 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f0f8b82a-6033-412c-892f-d66c1d82148f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:04:51.259754 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f0f8b82a-6033-412c-892f-d66c1d82148f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:51.268025 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1336/2662] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:04:51.273024 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367af601-b8a4-4d47-bcae-b0c689fcf9f4 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:51.283913 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367af601-b8a4-4d47-bcae-b0c689fcf9f4 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:51.289410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2320e5-e5c3-4e44-b287-76332431802f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:51.290103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2320e5-e5c3-4e44-b287-76332431802f admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:51.290406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2320e5-e5c3-4e44-b287-76332431802f admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:51.290685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2320e5-e5c3-4e44-b287-76332431802f admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:51.291430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2320e5-e5c3-4e44-b287-76332431802f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:04:51.291743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2320e5-e5c3-4e44-b287-76332431802f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:51.293424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2320e5-e5c3-4e44-b287-76332431802f admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:51.307379 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1327/2663] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:04:51.312269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8b82a-6033-412c-892f-d66c1d82148f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:51.322227 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8b82a-6033-412c-892f-d66c1d82148f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:51.327515 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf28a8c-8743-4b08-b4a5-b211c740cb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:51.328252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28a8c-8743-4b08-b4a5-b211c740cb3b admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:51.328536 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28a8c-8743-4b08-b4a5-b211c740cb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:51.328817 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28a8c-8743-4b08-b4a5-b211c740cb3b admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:51.329520 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf28a8c-8743-4b08-b4a5-b211c740cb3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:51.331459 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf28a8c-8743-4b08-b4a5-b211c740cb3b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:51.362328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1337/2664] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:04:51 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:04:51.366941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2320e5-e5c3-4e44-b287-76332431802f admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:51.377627 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2320e5-e5c3-4e44-b287-76332431802f admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:51.438462 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc810076-9f57-4700-893e-7e1f702dd40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:51.439149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc810076-9f57-4700-893e-7e1f702dd40c admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:51.439435 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc810076-9f57-4700-893e-7e1f702dd40c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:51.439716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc810076-9f57-4700-893e-7e1f702dd40c admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:51.440469 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc810076-9f57-4700-893e-7e1f702dd40c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:51.442427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc810076-9f57-4700-893e-7e1f702dd40c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:51.464859 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1328/2665] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:04:51 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:04:51.469130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf28a8c-8743-4b08-b4a5-b211c740cb3b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:51.480462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf28a8c-8743-4b08-b4a5-b211c740cb3b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:51.485580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfea341-09d3-4498-8cce-f97f23a2f87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:51.486384 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfea341-09d3-4498-8cce-f97f23a2f87e admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:51.486593 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfea341-09d3-4498-8cce-f97f23a2f87e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:51.486875 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfea341-09d3-4498-8cce-f97f23a2f87e admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:51.487615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfea341-09d3-4498-8cce-f97f23a2f87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:51.489269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfea341-09d3-4498-8cce-f97f23a2f87e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:51.500890 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1338/2666] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:04:51.505912 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc810076-9f57-4700-893e-7e1f702dd40c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:51.525078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc810076-9f57-4700-893e-7e1f702dd40c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:51.533029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7060420-89ec-4ab8-9770-7cb0eeb722a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:51.534147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7060420-89ec-4ab8-9770-7cb0eeb722a5 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:51.534655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7060420-89ec-4ab8-9770-7cb0eeb722a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:51.535169 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7060420-89ec-4ab8-9770-7cb0eeb722a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:51.536602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7060420-89ec-4ab8-9770-7cb0eeb722a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:51.540344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7060420-89ec-4ab8-9770-7cb0eeb722a5 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:51.560761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1329/2667] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:04:51.565331 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfea341-09d3-4498-8cce-f97f23a2f87e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:51.577191 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfea341-09d3-4498-8cce-f97f23a2f87e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:51.581916 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551de54-9fd9-46fe-ad8f-0f559f94dcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:51.582829 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551de54-9fd9-46fe-ad8f-0f559f94dcc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:51.583111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551de54-9fd9-46fe-ad8f-0f559f94dcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:51.583389 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551de54-9fd9-46fe-ad8f-0f559f94dcc4 admin admin] PATH_INFO: `/v3/projects/9c4ce652919f412b82c55e8f09d08de7/users/5db2ebbcc89e4e7f93ae5d65e42dae99/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:51.600203 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551de54-9fd9-46fe-ad8f-0f559f94dcc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4ce652919f412b82c55e8f09d08de7, user_id=5db2ebbcc89e4e7f93ae5d65e42dae99, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:51.602862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5551de54-9fd9-46fe-ad8f-0f559f94dcc4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:51.620285 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1339/2668] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:04:51 2022] PUT /identity/v3/projects/9c4ce652919f412b82c55e8f09d08de7/users/5db2ebbcc89e4e7f93ae5d65e42dae99/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:04:51.630769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e0449-df29-4423-891e-05340224305e None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:51.631057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e0449-df29-4423-891e-05340224305e None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:51.631370 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e0449-df29-4423-891e-05340224305e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:51.662797 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4c2e0449-df29-4423-891e-05340224305e None None] MFA Rules not processed for user `5db2ebbcc89e4e7f93ae5d65e42dae99`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:04:51.664134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4c2e0449-df29-4423-891e-05340224305e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:51.724519 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:04:51.745986 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1330/2669] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:04:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:04:51.751873 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551de54-9fd9-46fe-ad8f-0f559f94dcc4 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:51.811384 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:04:51.828635 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5551de54-9fd9-46fe-ad8f-0f559f94dcc4 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:51.833716 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12349e-6edf-473d-b2c7-066143438c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:51.834442 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12349e-6edf-473d-b2c7-066143438c2a admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:51.835346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12349e-6edf-473d-b2c7-066143438c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:51.835878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c12349e-6edf-473d-b2c7-066143438c2a admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:51.836698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12349e-6edf-473d-b2c7-066143438c2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:51.838831 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c12349e-6edf-473d-b2c7-066143438c2a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:51.868505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1340/2670] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:04:51 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:04:51.872445 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e0449-df29-4423-891e-05340224305e None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:51.886592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e0449-df29-4423-891e-05340224305e None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:51.953286 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d60951-5b81-4421-8bf8-feb7a0b30ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:51.953972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d60951-5b81-4421-8bf8-feb7a0b30ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:51.954253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d60951-5b81-4421-8bf8-feb7a0b30ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:51.954528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d60951-5b81-4421-8bf8-feb7a0b30ef6 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:51.955228 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d60951-5b81-4421-8bf8-feb7a0b30ef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:51.957210 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d60951-5b81-4421-8bf8-feb7a0b30ef6 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:51.978785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1331/2671] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:04:51 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:04:51.982844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12349e-6edf-473d-b2c7-066143438c2a admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:51.994239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c12349e-6edf-473d-b2c7-066143438c2a admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:51.999094 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5ade9-9b28-4fd2-bc87-04abc25feb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:51.999772 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5ade9-9b28-4fd2-bc87-04abc25feb57 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:52.000095 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5ade9-9b28-4fd2-bc87-04abc25feb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:52.000376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5ade9-9b28-4fd2-bc87-04abc25feb57 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:52.001116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5ade9-9b28-4fd2-bc87-04abc25feb57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:52.002739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5ade9-9b28-4fd2-bc87-04abc25feb57 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:52.015085 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1341/2672] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:04:52.019808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d60951-5b81-4421-8bf8-feb7a0b30ef6 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:52.030703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d60951-5b81-4421-8bf8-feb7a0b30ef6 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:52.035528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ac898-9f64-4687-8193-c817b02a88a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:52.036644 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ac898-9f64-4687-8193-c817b02a88a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:52.036839 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ac898-9f64-4687-8193-c817b02a88a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:52.037158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ac898-9f64-4687-8193-c817b02a88a3 admin admin] PATH_INFO: `/v3/projects/cc7a27b5c95549f0a510f52c4d839561/users/be3076577fa8455390ee0a45c9b744cd/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:52.054505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ac898-9f64-4687-8193-c817b02a88a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc7a27b5c95549f0a510f52c4d839561, user_id=be3076577fa8455390ee0a45c9b744cd, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:52.057311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ac898-9f64-4687-8193-c817b02a88a3 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:52.073518 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1332/2673] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:04:52 2022] PUT /identity/v3/projects/cc7a27b5c95549f0a510f52c4d839561/users/be3076577fa8455390ee0a45c9b744cd/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:04:52.078554 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5ade9-9b28-4fd2-bc87-04abc25feb57 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:52.089479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5ade9-9b28-4fd2-bc87-04abc25feb57 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:52.158738 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c291b-dcbd-4082-8504-5b9fcd45d83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:52.159347 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c291b-dcbd-4082-8504-5b9fcd45d83f admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:52.159637 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c291b-dcbd-4082-8504-5b9fcd45d83f admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:52.159944 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c291b-dcbd-4082-8504-5b9fcd45d83f admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:52.160706 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c291b-dcbd-4082-8504-5b9fcd45d83f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:52.162331 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09c291b-dcbd-4082-8504-5b9fcd45d83f admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:52.173081 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1342/2674] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:04:52.177678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ac898-9f64-4687-8193-c817b02a88a3 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:52.189163 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ac898-9f64-4687-8193-c817b02a88a3 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:52.194770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b219ac-927e-4ba3-a895-163c44d237b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:52.195679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b219ac-927e-4ba3-a895-163c44d237b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:52.195990 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b219ac-927e-4ba3-a895-163c44d237b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:52.196273 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b219ac-927e-4ba3-a895-163c44d237b2 admin admin] PATH_INFO: `/v3/projects/cc7a27b5c95549f0a510f52c4d839561/users/be3076577fa8455390ee0a45c9b744cd/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:52.198383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b219ac-927e-4ba3-a895-163c44d237b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc7a27b5c95549f0a510f52c4d839561, user_id=be3076577fa8455390ee0a45c9b744cd, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:52.201463 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b219ac-927e-4ba3-a895-163c44d237b2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:52.212825 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1333/2675] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:04:52 2022] PUT /identity/v3/projects/cc7a27b5c95549f0a510f52c4d839561/users/be3076577fa8455390ee0a45c9b744cd/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:04:52.232407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab55f0-7241-4135-91e1-57fee4907479 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:52.232857 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab55f0-7241-4135-91e1-57fee4907479 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:52.233289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ab55f0-7241-4135-91e1-57fee4907479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:52.264540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-c0ab55f0-7241-4135-91e1-57fee4907479 None None] MFA Rules not processed for user `be3076577fa8455390ee0a45c9b744cd`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:04:52.265858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-c0ab55f0-7241-4135-91e1-57fee4907479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:52.353675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1343/2676] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:04:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:04:52.370009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b219ac-927e-4ba3-a895-163c44d237b2 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:52.373004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-18b219ac-927e-4ba3-a895-163c44d237b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:04:52.441986 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b219ac-927e-4ba3-a895-163c44d237b2 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:52.446048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15d55d-d95e-4194-918b-69e5b5ce90fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:52.446725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15d55d-d95e-4194-918b-69e5b5ce90fd service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:52.447012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15d55d-d95e-4194-918b-69e5b5ce90fd service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:52.447321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15d55d-d95e-4194-918b-69e5b5ce90fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:52.458131 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15d55d-d95e-4194-918b-69e5b5ce90fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:52.459658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15d55d-d95e-4194-918b-69e5b5ce90fd service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:52.470811 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1334/2677] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:04:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:04:52.554883 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab55f0-7241-4135-91e1-57fee4907479 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:04:52.564675 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ab55f0-7241-4135-91e1-57fee4907479 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:04:52.568295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b7ec29-dd94-4ec0-aa36-21b438add017 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:04:52.568969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7ec29-dd94-4ec0-aa36-21b438add017 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:04:52.569256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7ec29-dd94-4ec0-aa36-21b438add017 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:04:52.569533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b7ec29-dd94-4ec0-aa36-21b438add017 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:04:52.613437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:04:52.630780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7ec29-dd94-4ec0-aa36-21b438add017 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:04:52.632354 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b7ec29-dd94-4ec0-aa36-21b438add017 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:04:52.642371 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1344/2678] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:04:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:05:02.465652 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:02.607759 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15d55d-d95e-4194-918b-69e5b5ce90fd service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:02.637817 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=8 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:02.671594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15d55d-d95e-4194-918b-69e5b5ce90fd service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:02.676818 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0a8a3-1ba3-4961-95b0-be12774bbc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:02.677777 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0a8a3-1ba3-4961-95b0-be12774bbc39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:02.678060 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0a8a3-1ba3-4961-95b0-be12774bbc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:02.678343 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf0a8a3-1ba3-4961-95b0-be12774bbc39 admin admin] PATH_INFO: `/v3/users/5db2ebbcc89e4e7f93ae5d65e42dae99` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:02.679665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0a8a3-1ba3-4961-95b0-be12774bbc39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5db2ebbcc89e4e7f93ae5d65e42dae99)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:02.682070 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf0a8a3-1ba3-4961-95b0-be12774bbc39 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:02.739753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-8bf0a8a3-1ba3-4961-95b0-be12774bbc39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5db2ebbcc89e4e7f93ae5d65e42dae99'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:02.753285 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-8bf0a8a3-1ba3-4961-95b0-be12774bbc39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5db2ebbcc89e4e7f93ae5d65e42dae99'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:02.754020 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-8bf0a8a3-1ba3-4961-95b0-be12774bbc39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5db2ebbcc89e4e7f93ae5d65e42dae99'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:02.761858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-8bf0a8a3-1ba3-4961-95b0-be12774bbc39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5db2ebbcc89e4e7f93ae5d65e42dae99'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:02.762577 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-8bf0a8a3-1ba3-4961-95b0-be12774bbc39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5db2ebbcc89e4e7f93ae5d65e42dae99'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:02.775564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1335/2679] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:05:02 2022] DELETE /identity/v3/users/5db2ebbcc89e4e7f93ae5d65e42dae99 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:02.780123 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ec9bb-910d-4d5a-bc5d-81e450093881 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:02.780488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ec9bb-910d-4d5a-bc5d-81e450093881 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:02.780840 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ec9bb-910d-4d5a-bc5d-81e450093881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:02.799511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-304ec9bb-910d-4d5a-bc5d-81e450093881 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:02.800773 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-304ec9bb-910d-4d5a-bc5d-81e450093881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:02.869472 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1345/2680] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:02.880077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0a8a3-1ba3-4961-95b0-be12774bbc39 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:02.881515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-8bf0a8a3-1ba3-4961-95b0-be12774bbc39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:02.943061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf0a8a3-1ba3-4961-95b0-be12774bbc39 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:02.946802 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38b885-bb8c-4571-a966-d0f698030090 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:02.947460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38b885-bb8c-4571-a966-d0f698030090 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:02.948030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38b885-bb8c-4571-a966-d0f698030090 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:02.948317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c38b885-bb8c-4571-a966-d0f698030090 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:02.959168 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c38b885-bb8c-4571-a966-d0f698030090 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:02.960770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c38b885-bb8c-4571-a966-d0f698030090 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:02.971052 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1336/2681] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:05:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:05:03.268385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ec9bb-910d-4d5a-bc5d-81e450093881 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:03.269923 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-304ec9bb-910d-4d5a-bc5d-81e450093881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:03.282477 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304ec9bb-910d-4d5a-bc5d-81e450093881 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:03.287326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:03.288246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:03.288533 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:03.288814 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] PATH_INFO: `/v3/projects/9c4ce652919f412b82c55e8f09d08de7` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:03.290053 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4ce652919f412b82c55e8f09d08de7)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:03.293344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:03.350605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4ce652919f412b82c55e8f09d08de7', 'request_id': 'req-1d378e44-6055-4810-b272-e334cf3c7086'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.357933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4ce652919f412b82c55e8f09d08de7', 'request_id': 'req-1d378e44-6055-4810-b272-e334cf3c7086'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.364684 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4ce652919f412b82c55e8f09d08de7', 'request_id': 'req-1d378e44-6055-4810-b272-e334cf3c7086'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.376862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] The token cache is being invalidate because project 9c4ce652919f412b82c55e8f09d08de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:05:03.377147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.377806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.379407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1346/2682] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:05:03 2022] DELETE /identity/v3/projects/9c4ce652919f412b82c55e8f09d08de7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:03.383832 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38b885-bb8c-4571-a966-d0f698030090 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:03.395439 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c38b885-bb8c-4571-a966-d0f698030090 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:03.415165 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:03.458023 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f0e44-a369-47b3-a416-0cd5e2344793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:03.458896 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f0e44-a369-47b3-a416-0cd5e2344793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:03.459174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f0e44-a369-47b3-a416-0cd5e2344793 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:03.459444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f0e44-a369-47b3-a416-0cd5e2344793 admin admin] PATH_INFO: `/v3/users/be3076577fa8455390ee0a45c9b744cd` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:03.460593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f0e44-a369-47b3-a416-0cd5e2344793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be3076577fa8455390ee0a45c9b744cd)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:03.462580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f0e44-a369-47b3-a416-0cd5e2344793 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:03.512433 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0b9f0e44-a369-47b3-a416-0cd5e2344793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be3076577fa8455390ee0a45c9b744cd'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.524305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0b9f0e44-a369-47b3-a416-0cd5e2344793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be3076577fa8455390ee0a45c9b744cd'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.524982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0b9f0e44-a369-47b3-a416-0cd5e2344793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be3076577fa8455390ee0a45c9b744cd'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.532262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0b9f0e44-a369-47b3-a416-0cd5e2344793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be3076577fa8455390ee0a45c9b744cd'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.532897 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0b9f0e44-a369-47b3-a416-0cd5e2344793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be3076577fa8455390ee0a45c9b744cd'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.545561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1337/2683] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:05:03 2022] DELETE /identity/v3/users/be3076577fa8455390ee0a45c9b744cd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:03.817110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:03.818976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:03.835580 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d378e44-6055-4810-b272-e334cf3c7086 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:03.895314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498b8776-27be-4859-bbda-ba89c0500b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:03.896283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b8776-27be-4859-bbda-ba89c0500b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:03.896570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b8776-27be-4859-bbda-ba89c0500b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:03.896847 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498b8776-27be-4859-bbda-ba89c0500b2c admin admin] PATH_INFO: `/v3/projects/cc7a27b5c95549f0a510f52c4d839561` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:03.898691 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b8776-27be-4859-bbda-ba89c0500b2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc7a27b5c95549f0a510f52c4d839561)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:03.900040 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498b8776-27be-4859-bbda-ba89c0500b2c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:03.942353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:03.960950 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-498b8776-27be-4859-bbda-ba89c0500b2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc7a27b5c95549f0a510f52c4d839561', 'request_id': 'req-498b8776-27be-4859-bbda-ba89c0500b2c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.969323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-498b8776-27be-4859-bbda-ba89c0500b2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc7a27b5c95549f0a510f52c4d839561', 'request_id': 'req-498b8776-27be-4859-bbda-ba89c0500b2c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.977349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-498b8776-27be-4859-bbda-ba89c0500b2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc7a27b5c95549f0a510f52c4d839561', 'request_id': 'req-498b8776-27be-4859-bbda-ba89c0500b2c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.989559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-498b8776-27be-4859-bbda-ba89c0500b2c admin admin] The token cache is being invalidate because project cc7a27b5c95549f0a510f52c4d839561 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:05:03.989785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-498b8776-27be-4859-bbda-ba89c0500b2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.990582 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-498b8776-27be-4859-bbda-ba89c0500b2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:03.992475 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1347/2684] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:05:03 2022] DELETE /identity/v3/projects/cc7a27b5c95549f0a510f52c4d839561 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:03.998205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c91b7-a168-4989-9b16-f9eb8188ae8e None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:03.998499 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c91b7-a168-4989-9b16-f9eb8188ae8e None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:03.998779 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c91b7-a168-4989-9b16-f9eb8188ae8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.017175 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-807c91b7-a168-4989-9b16-f9eb8188ae8e None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:04.018387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-807c91b7-a168-4989-9b16-f9eb8188ae8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:04.084493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1338/2685] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:04.092551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f518995-9459-4c54-88aa-848a5acc3f04 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:04.092841 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f518995-9459-4c54-88aa-848a5acc3f04 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:04.093117 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f518995-9459-4c54-88aa-848a5acc3f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.113714 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-9f518995-9459-4c54-88aa-848a5acc3f04 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:04.114986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-9f518995-9459-4c54-88aa-848a5acc3f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:04.124138 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1348/2686] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:04.128551 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c91b7-a168-4989-9b16-f9eb8188ae8e None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:04.138893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c91b7-a168-4989-9b16-f9eb8188ae8e None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:04.143670 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493c1b8-d75d-477e-949c-0c3a58114289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:04.144455 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493c1b8-d75d-477e-949c-0c3a58114289 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:04.145297 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493c1b8-d75d-477e-949c-0c3a58114289 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:04.145668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493c1b8-d75d-477e-949c-0c3a58114289 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.146396 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493c1b8-d75d-477e-949c-0c3a58114289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:05:04.146711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493c1b8-d75d-477e-949c-0c3a58114289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:04.148351 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8493c1b8-d75d-477e-949c-0c3a58114289 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:04.161570 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1339/2687] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 18:05:04.166123 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f518995-9459-4c54-88aa-848a5acc3f04 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:04.176226 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f518995-9459-4c54-88aa-848a5acc3f04 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:04.188210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d4608-e275-422f-bda0-719d0755cab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:04.188210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d4608-e275-422f-bda0-719d0755cab2 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:04.188210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d4608-e275-422f-bda0-719d0755cab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:04.188210 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d4608-e275-422f-bda0-719d0755cab2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.188920 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d4608-e275-422f-bda0-719d0755cab2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:04.188920 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d4608-e275-422f-bda0-719d0755cab2 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:04.214407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1349/2688] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:05:04 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:05:04.218611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493c1b8-d75d-477e-949c-0c3a58114289 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:04.229790 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493c1b8-d75d-477e-949c-0c3a58114289 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:04.258906 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:04.293443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e35559-eb10-4b6d-b1fe-72a0d2e4d178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:04.294633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e35559-eb10-4b6d-b1fe-72a0d2e4d178 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:04.294633 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e35559-eb10-4b6d-b1fe-72a0d2e4d178 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:04.294877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e35559-eb10-4b6d-b1fe-72a0d2e4d178 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.295388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e35559-eb10-4b6d-b1fe-72a0d2e4d178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:04.297499 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e35559-eb10-4b6d-b1fe-72a0d2e4d178 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:04.319381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1340/2689] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:05:04 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:05:04.324134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d4608-e275-422f-bda0-719d0755cab2 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:04.335734 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d4608-e275-422f-bda0-719d0755cab2 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:04.340682 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f8a36-7227-4d4c-9cc4-4acf568127cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:04.341369 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f8a36-7227-4d4c-9cc4-4acf568127cc admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:04.341692 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f8a36-7227-4d4c-9cc4-4acf568127cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:04.341975 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f8a36-7227-4d4c-9cc4-4acf568127cc admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.342717 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f8a36-7227-4d4c-9cc4-4acf568127cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:04.345065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f8a36-7227-4d4c-9cc4-4acf568127cc admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:04.360854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1350/2690] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:05:04.362310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e35559-eb10-4b6d-b1fe-72a0d2e4d178 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:04.373424 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e35559-eb10-4b6d-b1fe-72a0d2e4d178 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:04.378861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ee2c07-77c0-40a9-8642-52e686506c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:04.380460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee2c07-77c0-40a9-8642-52e686506c74 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:04.380785 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee2c07-77c0-40a9-8642-52e686506c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:04.381068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ee2c07-77c0-40a9-8642-52e686506c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.381922 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee2c07-77c0-40a9-8642-52e686506c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:04.383603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ee2c07-77c0-40a9-8642-52e686506c74 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:04.400809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1341/2691] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:05:04.403513 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f8a36-7227-4d4c-9cc4-4acf568127cc admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:04.414687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f8a36-7227-4d4c-9cc4-4acf568127cc admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:04.419702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b088d6-121f-4db0-8fb6-f8dd25f2f5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:04.420642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b088d6-121f-4db0-8fb6-f8dd25f2f5ca admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:04.420926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b088d6-121f-4db0-8fb6-f8dd25f2f5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:04.421201 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b088d6-121f-4db0-8fb6-f8dd25f2f5ca admin admin] PATH_INFO: `/v3/projects/eb28cb6a02204487b6ddbd6e86fc75ef/users/845a7e7c8e3449e39544fd8eb7309f13/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.439647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b088d6-121f-4db0-8fb6-f8dd25f2f5ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb28cb6a02204487b6ddbd6e86fc75ef, user_id=845a7e7c8e3449e39544fd8eb7309f13, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:04.442936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b088d6-121f-4db0-8fb6-f8dd25f2f5ca admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:04.460120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1351/2692] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:05:04 2022] PUT /identity/v3/projects/eb28cb6a02204487b6ddbd6e86fc75ef/users/845a7e7c8e3449e39544fd8eb7309f13/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:04.471410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abcb28-7971-4d9d-bda4-b39453d55798 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:04.471708 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abcb28-7971-4d9d-bda4-b39453d55798 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:04.472022 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abcb28-7971-4d9d-bda4-b39453d55798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.499752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-22abcb28-7971-4d9d-bda4-b39453d55798 None None] MFA Rules not processed for user `845a7e7c8e3449e39544fd8eb7309f13`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:04.501001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-22abcb28-7971-4d9d-bda4-b39453d55798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:04.569457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1342/2693] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:04.574231 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b088d6-121f-4db0-8fb6-f8dd25f2f5ca admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:04.639863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b088d6-121f-4db0-8fb6-f8dd25f2f5ca admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:04.645181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db0e1e-4996-44d3-a091-94c5cc6615e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:04.646020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db0e1e-4996-44d3-a091-94c5cc6615e1 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:04.646314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db0e1e-4996-44d3-a091-94c5cc6615e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:04.646592 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db0e1e-4996-44d3-a091-94c5cc6615e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.647279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db0e1e-4996-44d3-a091-94c5cc6615e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:04.649216 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db0e1e-4996-44d3-a091-94c5cc6615e1 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:04.678082 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1352/2694] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:05:04 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:05:04.679174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abcb28-7971-4d9d-bda4-b39453d55798 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:04.691154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abcb28-7971-4d9d-bda4-b39453d55798 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:04.750613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37143ec9-25fc-413b-8e1b-483d6cf6ee5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:04.751315 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37143ec9-25fc-413b-8e1b-483d6cf6ee5c admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:04.751598 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37143ec9-25fc-413b-8e1b-483d6cf6ee5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:04.751871 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37143ec9-25fc-413b-8e1b-483d6cf6ee5c admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.752605 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37143ec9-25fc-413b-8e1b-483d6cf6ee5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:04.755044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37143ec9-25fc-413b-8e1b-483d6cf6ee5c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:04.774414 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1343/2695] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:05:04 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:05:04.778810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db0e1e-4996-44d3-a091-94c5cc6615e1 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:04.789272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db0e1e-4996-44d3-a091-94c5cc6615e1 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:04.794157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782a414-0fc5-4b67-8fa4-13c1441ad12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:04.794848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782a414-0fc5-4b67-8fa4-13c1441ad12f admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:04.795133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782a414-0fc5-4b67-8fa4-13c1441ad12f admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:04.795416 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5782a414-0fc5-4b67-8fa4-13c1441ad12f admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.796183 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5782a414-0fc5-4b67-8fa4-13c1441ad12f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:04.797822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5782a414-0fc5-4b67-8fa4-13c1441ad12f admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:04.809327 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1353/2696] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:05:04.813371 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37143ec9-25fc-413b-8e1b-483d6cf6ee5c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:04.824136 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37143ec9-25fc-413b-8e1b-483d6cf6ee5c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:04.828837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5489cf80-b105-4625-90d7-b25049025b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:04.829495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489cf80-b105-4625-90d7-b25049025b62 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:04.829787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489cf80-b105-4625-90d7-b25049025b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:04.830067 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5489cf80-b105-4625-90d7-b25049025b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.830797 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489cf80-b105-4625-90d7-b25049025b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:04.832433 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5489cf80-b105-4625-90d7-b25049025b62 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:04.842678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1344/2697] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:05:04.846867 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782a414-0fc5-4b67-8fa4-13c1441ad12f admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:04.856983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5782a414-0fc5-4b67-8fa4-13c1441ad12f admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:04.861601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d542c2-2c01-45b2-a31c-cdc15aeccd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:04.862478 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d542c2-2c01-45b2-a31c-cdc15aeccd21 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:04.862763 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d542c2-2c01-45b2-a31c-cdc15aeccd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:04.863041 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d542c2-2c01-45b2-a31c-cdc15aeccd21 admin admin] PATH_INFO: `/v3/projects/066b6f92e0744f57a62489e68397e1b5/users/166d2f840c6a438598dd8a4137ea24e5/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.877806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d542c2-2c01-45b2-a31c-cdc15aeccd21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=066b6f92e0744f57a62489e68397e1b5, user_id=166d2f840c6a438598dd8a4137ea24e5, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:04.881085 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d542c2-2c01-45b2-a31c-cdc15aeccd21 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:04.890177 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:04.895270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1354/2698] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:05:04 2022] PUT /identity/v3/projects/066b6f92e0744f57a62489e68397e1b5/users/166d2f840c6a438598dd8a4137ea24e5/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:04.905922 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ebb87-7479-45de-8a1d-a237e2770b0b None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:04.906224 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ebb87-7479-45de-8a1d-a237e2770b0b None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:04.906524 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ebb87-7479-45de-8a1d-a237e2770b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:04.933506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-985ebb87-7479-45de-8a1d-a237e2770b0b None None] MFA Rules not processed for user `166d2f840c6a438598dd8a4137ea24e5`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:04.934848 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-985ebb87-7479-45de-8a1d-a237e2770b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:04.938984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:04.999843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1345/2699] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:05.009096 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d542c2-2c01-45b2-a31c-cdc15aeccd21 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:05.070162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d542c2-2c01-45b2-a31c-cdc15aeccd21 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:05.075275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bc5e8-12c5-461a-bd0d-80c96b52e0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:05.075869 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bc5e8-12c5-461a-bd0d-80c96b52e0d5 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:05.076182 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bc5e8-12c5-461a-bd0d-80c96b52e0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:05.076461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4bc5e8-12c5-461a-bd0d-80c96b52e0d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:05.077154 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bc5e8-12c5-461a-bd0d-80c96b52e0d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:05.079072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4bc5e8-12c5-461a-bd0d-80c96b52e0d5 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:05.104363 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1355/2700] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:05:05 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:05:05.108433 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ebb87-7479-45de-8a1d-a237e2770b0b None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:05.119682 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ebb87-7479-45de-8a1d-a237e2770b0b None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:05.176405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d4c1b-73cd-462d-bf68-40c7c5b3d3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:05.177084 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d4c1b-73cd-462d-bf68-40c7c5b3d3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:05.177373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d4c1b-73cd-462d-bf68-40c7c5b3d3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:05.177656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d4c1b-73cd-462d-bf68-40c7c5b3d3b9 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:05.178349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d4c1b-73cd-462d-bf68-40c7c5b3d3b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:05.180303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d4c1b-73cd-462d-bf68-40c7c5b3d3b9 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:05.202215 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1346/2701] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:05:05 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:05:05.206267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bc5e8-12c5-461a-bd0d-80c96b52e0d5 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:05.217205 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4bc5e8-12c5-461a-bd0d-80c96b52e0d5 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:05.221982 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e05d9-5401-4b35-8247-08131896b53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:05.222828 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e05d9-5401-4b35-8247-08131896b53f admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:05.223116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e05d9-5401-4b35-8247-08131896b53f admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:05.223401 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20e05d9-5401-4b35-8247-08131896b53f admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:05.224239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e05d9-5401-4b35-8247-08131896b53f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:05.226232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20e05d9-5401-4b35-8247-08131896b53f admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:05.236841 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1356/2702] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:05:05.241540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d4c1b-73cd-462d-bf68-40c7c5b3d3b9 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:05.251303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d4c1b-73cd-462d-bf68-40c7c5b3d3b9 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:05.256332 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13789930-9e6d-4685-a3a6-e00271974b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:05.257205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13789930-9e6d-4685-a3a6-e00271974b70 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:05.257486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13789930-9e6d-4685-a3a6-e00271974b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:05.257762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13789930-9e6d-4685-a3a6-e00271974b70 admin admin] PATH_INFO: `/v3/projects/53c807ee29e54d6f8115e28fc1ba488f/users/9967a976ae82453a9870a37e739c73dc/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:05.272935 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13789930-9e6d-4685-a3a6-e00271974b70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c807ee29e54d6f8115e28fc1ba488f, user_id=9967a976ae82453a9870a37e739c73dc, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:05.275565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13789930-9e6d-4685-a3a6-e00271974b70 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:05.288690 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1347/2703] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:05:05 2022] PUT /identity/v3/projects/53c807ee29e54d6f8115e28fc1ba488f/users/9967a976ae82453a9870a37e739c73dc/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:05.292671 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e05d9-5401-4b35-8247-08131896b53f admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:05.302891 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20e05d9-5401-4b35-8247-08131896b53f admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:05.360844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eebf27-31d9-42f6-97b3-64f09637555e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:05.361434 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eebf27-31d9-42f6-97b3-64f09637555e admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:05.361718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eebf27-31d9-42f6-97b3-64f09637555e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:05.361996 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eebf27-31d9-42f6-97b3-64f09637555e admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:05.362719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eebf27-31d9-42f6-97b3-64f09637555e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:05.364340 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eebf27-31d9-42f6-97b3-64f09637555e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:05.374014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1357/2704] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:05:05.378400 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13789930-9e6d-4685-a3a6-e00271974b70 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:05.389225 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13789930-9e6d-4685-a3a6-e00271974b70 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:05.393769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5a474-aae0-4a97-9023-5294a6cc919b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:05.395210 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5a474-aae0-4a97-9023-5294a6cc919b admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:05.395494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5a474-aae0-4a97-9023-5294a6cc919b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:05.395770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5a474-aae0-4a97-9023-5294a6cc919b admin admin] PATH_INFO: `/v3/projects/53c807ee29e54d6f8115e28fc1ba488f/users/9967a976ae82453a9870a37e739c73dc/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:05.397769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5a474-aae0-4a97-9023-5294a6cc919b admin admin] RBAC: Authorizing `identity:create_grant(project_id=53c807ee29e54d6f8115e28fc1ba488f, user_id=9967a976ae82453a9870a37e739c73dc, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:05.400422 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5a474-aae0-4a97-9023-5294a6cc919b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:05.412464 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1348/2705] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:05:05 2022] PUT /identity/v3/projects/53c807ee29e54d6f8115e28fc1ba488f/users/9967a976ae82453a9870a37e739c73dc/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:05:05.422597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6976df-db7c-441a-9386-ed2b73a09924 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:05.422896 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6976df-db7c-441a-9386-ed2b73a09924 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:05.423185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6976df-db7c-441a-9386-ed2b73a09924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:05.450840 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-7a6976df-db7c-441a-9386-ed2b73a09924 None None] MFA Rules not processed for user `9967a976ae82453a9870a37e739c73dc`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:05.452103 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7a6976df-db7c-441a-9386-ed2b73a09924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:05.511996 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:05.522739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1358/2706] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:05.532686 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5a474-aae0-4a97-9023-5294a6cc919b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:05.534476 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b9a5a474-aae0-4a97-9023-5294a6cc919b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:05.596307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5a474-aae0-4a97-9023-5294a6cc919b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:05.600190 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f12380-0ea2-427e-bbce-c602e676ccbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:05.600929 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f12380-0ea2-427e-bbce-c602e676ccbd service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:05.601153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f12380-0ea2-427e-bbce-c602e676ccbd service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:05.601483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f12380-0ea2-427e-bbce-c602e676ccbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:05.613579 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f12380-0ea2-427e-bbce-c602e676ccbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:05.614961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f12380-0ea2-427e-bbce-c602e676ccbd service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:05.620308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:05.626829 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1349/2707] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:05:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:05:06.157813 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6976df-db7c-441a-9386-ed2b73a09924 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:06.159263 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7a6976df-db7c-441a-9386-ed2b73a09924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:06.172951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6976df-db7c-441a-9386-ed2b73a09924 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:06.177308 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cd750-ca0d-4929-9349-2a64145a5566 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:06.177996 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cd750-ca0d-4929-9349-2a64145a5566 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:06.178268 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cd750-ca0d-4929-9349-2a64145a5566 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:06.178546 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04cd750-ca0d-4929-9349-2a64145a5566 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:06.242273 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cd750-ca0d-4929-9349-2a64145a5566 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:06.243722 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04cd750-ca0d-4929-9349-2a64145a5566 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:06.253287 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1359/2708] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:05:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:05:10.261914 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f12380-0ea2-427e-bbce-c602e676ccbd service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:10.270516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f12380-0ea2-427e-bbce-c602e676ccbd service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:10.274170 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3df39b-e28a-48a8-85bf-7850bf9b3648 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:10.274822 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3df39b-e28a-48a8-85bf-7850bf9b3648 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:10.275109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3df39b-e28a-48a8-85bf-7850bf9b3648 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:10.275391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3df39b-e28a-48a8-85bf-7850bf9b3648 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:10.334880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3df39b-e28a-48a8-85bf-7850bf9b3648 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:10.336278 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3df39b-e28a-48a8-85bf-7850bf9b3648 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:10.346100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1350/2709] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:05:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:05:12.418507 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cd750-ca0d-4929-9349-2a64145a5566 service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:12.480247 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04cd750-ca0d-4929-9349-2a64145a5566 service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:12.484671 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011de0ad-bc8f-479e-9871-52c8ab492bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:12.485538 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011de0ad-bc8f-479e-9871-52c8ab492bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:12.485818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011de0ad-bc8f-479e-9871-52c8ab492bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:12.486093 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011de0ad-bc8f-479e-9871-52c8ab492bbd admin admin] PATH_INFO: `/v3/users/845a7e7c8e3449e39544fd8eb7309f13` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:12.487271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011de0ad-bc8f-479e-9871-52c8ab492bbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=845a7e7c8e3449e39544fd8eb7309f13)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:12.490408 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011de0ad-bc8f-479e-9871-52c8ab492bbd admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:12.541363 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-011de0ad-bc8f-479e-9871-52c8ab492bbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '845a7e7c8e3449e39544fd8eb7309f13'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:12.553405 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-011de0ad-bc8f-479e-9871-52c8ab492bbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '845a7e7c8e3449e39544fd8eb7309f13'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:12.561051 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-011de0ad-bc8f-479e-9871-52c8ab492bbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '845a7e7c8e3449e39544fd8eb7309f13'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:12.561762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-011de0ad-bc8f-479e-9871-52c8ab492bbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '845a7e7c8e3449e39544fd8eb7309f13'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:12.569280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-011de0ad-bc8f-479e-9871-52c8ab492bbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '845a7e7c8e3449e39544fd8eb7309f13'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:12.575518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1360/2710] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:05:12 2022] DELETE /identity/v3/users/845a7e7c8e3449e39544fd8eb7309f13 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:12.580107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f231a1a-ba88-465d-8b7d-41e361fba5df None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:12.580606 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f231a1a-ba88-465d-8b7d-41e361fba5df None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:12.580715 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f231a1a-ba88-465d-8b7d-41e361fba5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:12.597800 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-7f231a1a-ba88-465d-8b7d-41e361fba5df None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:12.598919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7f231a1a-ba88-465d-8b7d-41e361fba5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:12.656258 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1351/2711] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:12.666417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011de0ad-bc8f-479e-9871-52c8ab492bbd admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:12.667817 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-011de0ad-bc8f-479e-9871-52c8ab492bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:12.681981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:12.727152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011de0ad-bc8f-479e-9871-52c8ab492bbd admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:12.730920 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64e674-b893-482b-9081-bd9d709dcc74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:12.731567 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64e674-b893-482b-9081-bd9d709dcc74 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:12.731852 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64e674-b893-482b-9081-bd9d709dcc74 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:12.732157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64e674-b893-482b-9081-bd9d709dcc74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:12.741206 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64e674-b893-482b-9081-bd9d709dcc74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:12.742632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64e674-b893-482b-9081-bd9d709dcc74 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:12.752007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1361/2712] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:05:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:05:13.019996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f231a1a-ba88-465d-8b7d-41e361fba5df None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:13.021443 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7f231a1a-ba88-465d-8b7d-41e361fba5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:13.039312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f231a1a-ba88-465d-8b7d-41e361fba5df None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:13.044418 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:13.045317 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:13.045624 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:13.045886 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] PATH_INFO: `/v3/projects/eb28cb6a02204487b6ddbd6e86fc75ef` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:13.047123 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb28cb6a02204487b6ddbd6e86fc75ef)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:13.049091 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:13.098285 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb28cb6a02204487b6ddbd6e86fc75ef', 'request_id': 'req-27f64b70-4668-4ab7-a3c1-db87edafcca1'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.106465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb28cb6a02204487b6ddbd6e86fc75ef', 'request_id': 'req-27f64b70-4668-4ab7-a3c1-db87edafcca1'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.112220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb28cb6a02204487b6ddbd6e86fc75ef', 'request_id': 'req-27f64b70-4668-4ab7-a3c1-db87edafcca1'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.124124 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] The token cache is being invalidate because project eb28cb6a02204487b6ddbd6e86fc75ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:05:13.124413 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.125117 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.126665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1352/2713] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:05:13 2022] DELETE /identity/v3/projects/eb28cb6a02204487b6ddbd6e86fc75ef => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:13.131558 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64e674-b893-482b-9081-bd9d709dcc74 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:13.141816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64e674-b893-482b-9081-bd9d709dcc74 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:13.198870 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0de61-bdcb-4f7b-aca4-227af2d323b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:13.199765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0de61-bdcb-4f7b-aca4-227af2d323b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:13.200090 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0de61-bdcb-4f7b-aca4-227af2d323b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:13.200375 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0de61-bdcb-4f7b-aca4-227af2d323b7 admin admin] PATH_INFO: `/v3/users/166d2f840c6a438598dd8a4137ea24e5` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:13.201586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0de61-bdcb-4f7b-aca4-227af2d323b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=166d2f840c6a438598dd8a4137ea24e5)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:13.203532 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0de61-bdcb-4f7b-aca4-227af2d323b7 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:13.250329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1fe0de61-bdcb-4f7b-aca4-227af2d323b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '166d2f840c6a438598dd8a4137ea24e5'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.262786 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1fe0de61-bdcb-4f7b-aca4-227af2d323b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '166d2f840c6a438598dd8a4137ea24e5'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.271045 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1fe0de61-bdcb-4f7b-aca4-227af2d323b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '166d2f840c6a438598dd8a4137ea24e5'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.271842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1fe0de61-bdcb-4f7b-aca4-227af2d323b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '166d2f840c6a438598dd8a4137ea24e5'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.280090 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-1fe0de61-bdcb-4f7b-aca4-227af2d323b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '166d2f840c6a438598dd8a4137ea24e5'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.287471 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1362/2714] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:05:13 2022] DELETE /identity/v3/users/166d2f840c6a438598dd8a4137ea24e5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:05:13.577931 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:13.579394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:13.594151 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f64b70-4668-4ab7-a3c1-db87edafcca1 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:13.613730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:13.655490 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:13.656307 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:13.656585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:13.656856 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] PATH_INFO: `/v3/projects/066b6f92e0744f57a62489e68397e1b5` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:13.658038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=066b6f92e0744f57a62489e68397e1b5)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:13.660034 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:13.712035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '066b6f92e0744f57a62489e68397e1b5', 'request_id': 'req-44d67b0c-c613-47b6-aa1a-cf019a9604ee'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.720077 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '066b6f92e0744f57a62489e68397e1b5', 'request_id': 'req-44d67b0c-c613-47b6-aa1a-cf019a9604ee'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.725993 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '066b6f92e0744f57a62489e68397e1b5', 'request_id': 'req-44d67b0c-c613-47b6-aa1a-cf019a9604ee'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.737283 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] The token cache is being invalidate because project 066b6f92e0744f57a62489e68397e1b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:05:13.738099 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.738771 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.740366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1353/2715] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:05:13 2022] DELETE /identity/v3/projects/066b6f92e0744f57a62489e68397e1b5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:13.744491 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0de61-bdcb-4f7b-aca4-227af2d323b7 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:13.745949 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1fe0de61-bdcb-4f7b-aca4-227af2d323b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:13.759617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0de61-bdcb-4f7b-aca4-227af2d323b7 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:13.798379 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:13.829770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9d13a-9e8e-4880-b7f0-794b078f967f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:13.830672 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9d13a-9e8e-4880-b7f0-794b078f967f admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:13.830959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9d13a-9e8e-4880-b7f0-794b078f967f admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:13.831246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9d13a-9e8e-4880-b7f0-794b078f967f admin admin] PATH_INFO: `/v3/users/9967a976ae82453a9870a37e739c73dc` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:13.832484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9d13a-9e8e-4880-b7f0-794b078f967f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9967a976ae82453a9870a37e739c73dc)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:13.834441 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c9d13a-9e8e-4880-b7f0-794b078f967f admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:13.887729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-56c9d13a-9e8e-4880-b7f0-794b078f967f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9967a976ae82453a9870a37e739c73dc'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.911816 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-56c9d13a-9e8e-4880-b7f0-794b078f967f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9967a976ae82453a9870a37e739c73dc'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.919866 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-56c9d13a-9e8e-4880-b7f0-794b078f967f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9967a976ae82453a9870a37e739c73dc'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.920550 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-56c9d13a-9e8e-4880-b7f0-794b078f967f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9967a976ae82453a9870a37e739c73dc'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.928572 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-56c9d13a-9e8e-4880-b7f0-794b078f967f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9967a976ae82453a9870a37e739c73dc'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:13.935615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1363/2716] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:05:13 2022] DELETE /identity/v3/users/9967a976ae82453a9870a37e739c73dc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:05:14.258701 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:14.260209 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:14.273189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d67b0c-c613-47b6-aa1a-cf019a9604ee admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:14.341571 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dee1c9-5b5f-4215-8f42-e6471beb7482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:14.342496 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dee1c9-5b5f-4215-8f42-e6471beb7482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:14.342784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dee1c9-5b5f-4215-8f42-e6471beb7482 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:14.343116 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dee1c9-5b5f-4215-8f42-e6471beb7482 admin admin] PATH_INFO: `/v3/projects/53c807ee29e54d6f8115e28fc1ba488f` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:14.344506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dee1c9-5b5f-4215-8f42-e6471beb7482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53c807ee29e54d6f8115e28fc1ba488f)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:14.346478 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dee1c9-5b5f-4215-8f42-e6471beb7482 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:14.395366 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c7dee1c9-5b5f-4215-8f42-e6471beb7482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53c807ee29e54d6f8115e28fc1ba488f', 'request_id': 'req-c7dee1c9-5b5f-4215-8f42-e6471beb7482'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:14.398948 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:14.404113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c7dee1c9-5b5f-4215-8f42-e6471beb7482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53c807ee29e54d6f8115e28fc1ba488f', 'request_id': 'req-c7dee1c9-5b5f-4215-8f42-e6471beb7482'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:14.411505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c7dee1c9-5b5f-4215-8f42-e6471beb7482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53c807ee29e54d6f8115e28fc1ba488f', 'request_id': 'req-c7dee1c9-5b5f-4215-8f42-e6471beb7482'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:14.423950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c7dee1c9-5b5f-4215-8f42-e6471beb7482 admin admin] The token cache is being invalidate because project 53c807ee29e54d6f8115e28fc1ba488f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:05:14.424255 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c7dee1c9-5b5f-4215-8f42-e6471beb7482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:14.425515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c7dee1c9-5b5f-4215-8f42-e6471beb7482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:14.427078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1354/2717] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:05:14 2022] DELETE /identity/v3/projects/53c807ee29e54d6f8115e28fc1ba488f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:14.433257 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47dc01-1bab-4aad-83a7-fffbc32d198e None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:14.433784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47dc01-1bab-4aad-83a7-fffbc32d198e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:14.434072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47dc01-1bab-4aad-83a7-fffbc32d198e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:14.453506 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-8f47dc01-1bab-4aad-83a7-fffbc32d198e None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:14.454868 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8f47dc01-1bab-4aad-83a7-fffbc32d198e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:14.517524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1364/2718] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:14.528437 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc9e09-017b-402c-9fed-98f04d427b56 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:14.528725 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc9e09-017b-402c-9fed-98f04d427b56 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:14.529004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc9e09-017b-402c-9fed-98f04d427b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:14.547918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-3afc9e09-017b-402c-9fed-98f04d427b56 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:14.549134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3afc9e09-017b-402c-9fed-98f04d427b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:14.557029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1355/2719] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:14.561326 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47dc01-1bab-4aad-83a7-fffbc32d198e None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:14.571579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47dc01-1bab-4aad-83a7-fffbc32d198e None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:14.576198 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381b521-1430-4f06-aaa0-2c34d17508b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:14.576859 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381b521-1430-4f06-aaa0-2c34d17508b8 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:14.577141 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381b521-1430-4f06-aaa0-2c34d17508b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:14.577415 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381b521-1430-4f06-aaa0-2c34d17508b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:14.578137 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381b521-1430-4f06-aaa0-2c34d17508b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:05:14.578443 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381b521-1430-4f06-aaa0-2c34d17508b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:14.580072 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381b521-1430-4f06-aaa0-2c34d17508b8 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:14.592635 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1365/2720] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:05:14.597764 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc9e09-017b-402c-9fed-98f04d427b56 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:14.609423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc9e09-017b-402c-9fed-98f04d427b56 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:14.614212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fca93f-b2c1-43dc-beeb-dd4a40f03e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:14.614880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fca93f-b2c1-43dc-beeb-dd4a40f03e61 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:14.615167 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fca93f-b2c1-43dc-beeb-dd4a40f03e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:14.615446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fca93f-b2c1-43dc-beeb-dd4a40f03e61 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:14.616174 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fca93f-b2c1-43dc-beeb-dd4a40f03e61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:14.618088 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fca93f-b2c1-43dc-beeb-dd4a40f03e61 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:14.644449 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1356/2721] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:05:14 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:05:14.648496 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381b521-1430-4f06-aaa0-2c34d17508b8 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:14.658535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381b521-1430-4f06-aaa0-2c34d17508b8 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:14.699624 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:14.717606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07f866-7af4-4afc-a5f6-11671fcd2441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:14.718286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07f866-7af4-4afc-a5f6-11671fcd2441 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:14.718571 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07f866-7af4-4afc-a5f6-11671fcd2441 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:14.718851 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f07f866-7af4-4afc-a5f6-11671fcd2441 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:14.719547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07f866-7af4-4afc-a5f6-11671fcd2441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:14.721501 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f07f866-7af4-4afc-a5f6-11671fcd2441 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:14.744039 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1366/2722] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:05:14 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:05:14.748311 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fca93f-b2c1-43dc-beeb-dd4a40f03e61 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:14.759522 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fca93f-b2c1-43dc-beeb-dd4a40f03e61 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:14.764566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1f1a8-7516-4878-9ce1-86454f25f5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:14.765300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1f1a8-7516-4878-9ce1-86454f25f5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:14.765598 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1f1a8-7516-4878-9ce1-86454f25f5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:14.765876 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1f1a8-7516-4878-9ce1-86454f25f5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:14.766618 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1f1a8-7516-4878-9ce1-86454f25f5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:14.768268 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1f1a8-7516-4878-9ce1-86454f25f5c0 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:14.780488 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1357/2723] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:05:14.785385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07f866-7af4-4afc-a5f6-11671fcd2441 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:14.796066 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f07f866-7af4-4afc-a5f6-11671fcd2441 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:14.801030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7d0d0-6889-4fdc-be50-4b10062be796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:14.801712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d0d0-6889-4fdc-be50-4b10062be796 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:14.801992 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d0d0-6889-4fdc-be50-4b10062be796 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:14.802272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d0d0-6889-4fdc-be50-4b10062be796 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:14.803001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7d0d0-6889-4fdc-be50-4b10062be796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:14.804610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7d0d0-6889-4fdc-be50-4b10062be796 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:14.815747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1367/2724] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:05:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:05:14.820025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1f1a8-7516-4878-9ce1-86454f25f5c0 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:14.830620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1f1a8-7516-4878-9ce1-86454f25f5c0 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:14.836093 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ce53a-8892-4cc2-acf2-3964499bf56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:14.837026 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ce53a-8892-4cc2-acf2-3964499bf56c admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:14.837313 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ce53a-8892-4cc2-acf2-3964499bf56c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:14.837592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99ce53a-8892-4cc2-acf2-3964499bf56c admin admin] PATH_INFO: `/v3/projects/78b2429a518547b39764c86d73005924/users/1c4e1be2a9004a55999504f1bd8a9785/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:14.853249 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ce53a-8892-4cc2-acf2-3964499bf56c admin admin] RBAC: Authorizing `identity:create_grant(project_id=78b2429a518547b39764c86d73005924, user_id=1c4e1be2a9004a55999504f1bd8a9785, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:14.855934 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99ce53a-8892-4cc2-acf2-3964499bf56c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:14.871752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1358/2725] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:05:14 2022] PUT /identity/v3/projects/78b2429a518547b39764c86d73005924/users/1c4e1be2a9004a55999504f1bd8a9785/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 18:05:14.884556 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4ae9e-fe32-4fda-a057-bd713610eab3 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:14.884859 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4ae9e-fe32-4fda-a057-bd713610eab3 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:14.885149 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4ae9e-fe32-4fda-a057-bd713610eab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:14.913461 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-feb4ae9e-fe32-4fda-a057-bd713610eab3 None None] MFA Rules not processed for user `1c4e1be2a9004a55999504f1bd8a9785`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:14.914704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-feb4ae9e-fe32-4fda-a057-bd713610eab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:14.982277 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1368/2726] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:14.986636 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ce53a-8892-4cc2-acf2-3964499bf56c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:15.048486 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99ce53a-8892-4cc2-acf2-3964499bf56c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:15.053840 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c5df9-6726-4110-a62c-a24237c10fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:15.054515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c5df9-6726-4110-a62c-a24237c10fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:15.054820 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c5df9-6726-4110-a62c-a24237c10fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:15.055107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66c5df9-6726-4110-a62c-a24237c10fe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:15.055805 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c5df9-6726-4110-a62c-a24237c10fe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:15.057752 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66c5df9-6726-4110-a62c-a24237c10fe7 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:15.086045 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1359/2727] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:05:14 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:05:15.089975 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4ae9e-fe32-4fda-a057-bd713610eab3 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:15.101498 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4ae9e-fe32-4fda-a057-bd713610eab3 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:15.159225 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91dd94e-e53b-49b2-afd4-7d8f795a123d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:15.159908 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dd94e-e53b-49b2-afd4-7d8f795a123d admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:15.160333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dd94e-e53b-49b2-afd4-7d8f795a123d admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:15.160628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91dd94e-e53b-49b2-afd4-7d8f795a123d admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:15.161313 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91dd94e-e53b-49b2-afd4-7d8f795a123d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:15.163237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91dd94e-e53b-49b2-afd4-7d8f795a123d admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:15.183526 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1369/2728] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:05:15 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:05:15.187506 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c5df9-6726-4110-a62c-a24237c10fe7 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:15.198665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66c5df9-6726-4110-a62c-a24237c10fe7 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:15.203522 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d0f47-91f2-495f-8d40-d92e5fc92b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:15.204275 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d0f47-91f2-495f-8d40-d92e5fc92b25 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:15.204563 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d0f47-91f2-495f-8d40-d92e5fc92b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:15.204842 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d0f47-91f2-495f-8d40-d92e5fc92b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:15.205586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d0f47-91f2-495f-8d40-d92e5fc92b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:15.207761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d0f47-91f2-495f-8d40-d92e5fc92b25 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:15.219274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1360/2729] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:05:15.223644 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91dd94e-e53b-49b2-afd4-7d8f795a123d admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:15.233963 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91dd94e-e53b-49b2-afd4-7d8f795a123d admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:15.238959 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e900be-4da6-4216-8f0e-b669b7bb0b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:15.239842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e900be-4da6-4216-8f0e-b669b7bb0b4b admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:15.240162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e900be-4da6-4216-8f0e-b669b7bb0b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:15.240445 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e900be-4da6-4216-8f0e-b669b7bb0b4b admin admin] PATH_INFO: `/v3/projects/886cd133545d4e4e920c3b728e6451ab/users/4761208f3d1a475ab4f3a94b2f5ea2a9/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:15.258111 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e900be-4da6-4216-8f0e-b669b7bb0b4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=886cd133545d4e4e920c3b728e6451ab, user_id=4761208f3d1a475ab4f3a94b2f5ea2a9, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:15.260815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e900be-4da6-4216-8f0e-b669b7bb0b4b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:15.275152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1370/2730] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:05:15 2022] PUT /identity/v3/projects/886cd133545d4e4e920c3b728e6451ab/users/4761208f3d1a475ab4f3a94b2f5ea2a9/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:15.279180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d0f47-91f2-495f-8d40-d92e5fc92b25 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:15.289720 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d0f47-91f2-495f-8d40-d92e5fc92b25 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:15.337552 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:15.349414 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f63d38-b2b8-481a-82b7-7469a5078a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:15.350086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f63d38-b2b8-481a-82b7-7469a5078a84 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:15.350373 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f63d38-b2b8-481a-82b7-7469a5078a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:15.350648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f63d38-b2b8-481a-82b7-7469a5078a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:15.351376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f63d38-b2b8-481a-82b7-7469a5078a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:15.352989 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f63d38-b2b8-481a-82b7-7469a5078a84 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:15.363561 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1361/2731] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:05:15.367699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e900be-4da6-4216-8f0e-b669b7bb0b4b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:15.378695 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e900be-4da6-4216-8f0e-b669b7bb0b4b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:15.383373 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001b1ae-b8fb-4ee7-844d-b9ef50f12fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:15.384300 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001b1ae-b8fb-4ee7-844d-b9ef50f12fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:15.384610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001b1ae-b8fb-4ee7-844d-b9ef50f12fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:15.384886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001b1ae-b8fb-4ee7-844d-b9ef50f12fc7 admin admin] PATH_INFO: `/v3/projects/886cd133545d4e4e920c3b728e6451ab/users/4761208f3d1a475ab4f3a94b2f5ea2a9/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:15.386878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001b1ae-b8fb-4ee7-844d-b9ef50f12fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=886cd133545d4e4e920c3b728e6451ab, user_id=4761208f3d1a475ab4f3a94b2f5ea2a9, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:15.389543 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001b1ae-b8fb-4ee7-844d-b9ef50f12fc7 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:15.400856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1371/2732] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:05:15 2022] PUT /identity/v3/projects/886cd133545d4e4e920c3b728e6451ab/users/4761208f3d1a475ab4f3a94b2f5ea2a9/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:15.412454 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba010923-46c7-41f4-bbcb-db6ca281b81c None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:15.412743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba010923-46c7-41f4-bbcb-db6ca281b81c None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:15.413030 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba010923-46c7-41f4-bbcb-db6ca281b81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:15.439349 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-ba010923-46c7-41f4-bbcb-db6ca281b81c None None] MFA Rules not processed for user `4761208f3d1a475ab4f3a94b2f5ea2a9`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:15.441475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ba010923-46c7-41f4-bbcb-db6ca281b81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:15.513692 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1362/2733] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:15.525487 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001b1ae-b8fb-4ee7-844d-b9ef50f12fc7 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:15.527385 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7001b1ae-b8fb-4ee7-844d-b9ef50f12fc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:15.532866 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:15.588306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001b1ae-b8fb-4ee7-844d-b9ef50f12fc7 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:15.591917 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23922b-c2fb-464e-bc56-7be37369723c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:15.592593 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23922b-c2fb-464e-bc56-7be37369723c service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:15.592878 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23922b-c2fb-464e-bc56-7be37369723c service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:15.593165 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c23922b-c2fb-464e-bc56-7be37369723c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:15.651423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23922b-c2fb-464e-bc56-7be37369723c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:15.652909 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c23922b-c2fb-464e-bc56-7be37369723c service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:15.662893 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1372/2734] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:05:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:05:19.825920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba010923-46c7-41f4-bbcb-db6ca281b81c None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:19.827344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ba010923-46c7-41f4-bbcb-db6ca281b81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:19.842218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba010923-46c7-41f4-bbcb-db6ca281b81c None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:19.846939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc7138-feff-4585-8840-1a4e0af52ea8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:19.847593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc7138-feff-4585-8840-1a4e0af52ea8 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:19.847873 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc7138-feff-4585-8840-1a4e0af52ea8 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:19.848188 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc7138-feff-4585-8840-1a4e0af52ea8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:19.858097 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc7138-feff-4585-8840-1a4e0af52ea8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:19.859528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc7138-feff-4585-8840-1a4e0af52ea8 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:19.869430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1363/2735] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:05:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:05:24.728788 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23922b-c2fb-464e-bc56-7be37369723c service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:24.794277 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c23922b-c2fb-464e-bc56-7be37369723c service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:24.799437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32b949-4989-4c00-86ce-930039918299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:24.800349 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32b949-4989-4c00-86ce-930039918299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:24.800642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32b949-4989-4c00-86ce-930039918299 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:24.800923 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b32b949-4989-4c00-86ce-930039918299 admin admin] PATH_INFO: `/v3/users/1c4e1be2a9004a55999504f1bd8a9785` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:24.802142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32b949-4989-4c00-86ce-930039918299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c4e1be2a9004a55999504f1bd8a9785)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:24.804108 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b32b949-4989-4c00-86ce-930039918299 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:24.852055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7b32b949-4989-4c00-86ce-930039918299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c4e1be2a9004a55999504f1bd8a9785'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:24.860070 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:24.866879 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7b32b949-4989-4c00-86ce-930039918299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c4e1be2a9004a55999504f1bd8a9785'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:24.876280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7b32b949-4989-4c00-86ce-930039918299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c4e1be2a9004a55999504f1bd8a9785'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:24.876924 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7b32b949-4989-4c00-86ce-930039918299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c4e1be2a9004a55999504f1bd8a9785'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:24.884339 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-7b32b949-4989-4c00-86ce-930039918299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c4e1be2a9004a55999504f1bd8a9785'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:24.890583 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1373/2736] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:05:24 2022] DELETE /identity/v3/users/1c4e1be2a9004a55999504f1bd8a9785 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:24.895130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf687cd-4b4a-45d4-b5fc-8a803bde7629 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:24.895621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf687cd-4b4a-45d4-b5fc-8a803bde7629 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:24.895761 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf687cd-4b4a-45d4-b5fc-8a803bde7629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:24.914352 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-1bf687cd-4b4a-45d4-b5fc-8a803bde7629 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:24.915574 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1bf687cd-4b4a-45d4-b5fc-8a803bde7629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:24.986007 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1364/2737] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:24.997950 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32b949-4989-4c00-86ce-930039918299 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:24.999454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7b32b949-4989-4c00-86ce-930039918299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:25.066287 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b32b949-4989-4c00-86ce-930039918299 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:25.070509 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a398968-ebcf-4cfa-ade4-4287935dd8a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:25.071181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a398968-ebcf-4cfa-ade4-4287935dd8a4 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:25.071464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a398968-ebcf-4cfa-ade4-4287935dd8a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:25.071777 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a398968-ebcf-4cfa-ade4-4287935dd8a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:25.082517 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a398968-ebcf-4cfa-ade4-4287935dd8a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:25.084004 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a398968-ebcf-4cfa-ade4-4287935dd8a4 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:25.098574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1374/2738] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:05:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:05:25.388190 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf687cd-4b4a-45d4-b5fc-8a803bde7629 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:25.389656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1bf687cd-4b4a-45d4-b5fc-8a803bde7629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:25.402975 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf687cd-4b4a-45d4-b5fc-8a803bde7629 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:25.408357 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:25.409253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:25.409626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:25.409809 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] PATH_INFO: `/v3/projects/78b2429a518547b39764c86d73005924` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:25.411107 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78b2429a518547b39764c86d73005924)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:25.413096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:25.463671 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78b2429a518547b39764c86d73005924', 'request_id': 'req-2ba0846f-4350-4420-aefe-6ec192079331'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:25.471892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78b2429a518547b39764c86d73005924', 'request_id': 'req-2ba0846f-4350-4420-aefe-6ec192079331'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:25.473354 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:25.479648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78b2429a518547b39764c86d73005924', 'request_id': 'req-2ba0846f-4350-4420-aefe-6ec192079331'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:25.491835 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] The token cache is being invalidate because project 78b2429a518547b39764c86d73005924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:05:25.492062 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:25.492736 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:25.494302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1365/2739] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:05:25 2022] DELETE /identity/v3/projects/78b2429a518547b39764c86d73005924 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:25.498431 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a398968-ebcf-4cfa-ade4-4287935dd8a4 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:25.508977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a398968-ebcf-4cfa-ade4-4287935dd8a4 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:25.570796 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93928e0e-1d4c-4e55-9cb0-968bfa911922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:25.571673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93928e0e-1d4c-4e55-9cb0-968bfa911922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:25.571985 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93928e0e-1d4c-4e55-9cb0-968bfa911922 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:25.572263 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93928e0e-1d4c-4e55-9cb0-968bfa911922 admin admin] PATH_INFO: `/v3/users/4761208f3d1a475ab4f3a94b2f5ea2a9` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:25.573536 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93928e0e-1d4c-4e55-9cb0-968bfa911922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4761208f3d1a475ab4f3a94b2f5ea2a9)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:25.575465 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93928e0e-1d4c-4e55-9cb0-968bfa911922 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:25.624784 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-93928e0e-1d4c-4e55-9cb0-968bfa911922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4761208f3d1a475ab4f3a94b2f5ea2a9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:25.637962 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-93928e0e-1d4c-4e55-9cb0-968bfa911922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4761208f3d1a475ab4f3a94b2f5ea2a9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:25.646248 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-93928e0e-1d4c-4e55-9cb0-968bfa911922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4761208f3d1a475ab4f3a94b2f5ea2a9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:25.646912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-93928e0e-1d4c-4e55-9cb0-968bfa911922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4761208f3d1a475ab4f3a94b2f5ea2a9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:25.655001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-93928e0e-1d4c-4e55-9cb0-968bfa911922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4761208f3d1a475ab4f3a94b2f5ea2a9'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:25.656590 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:25.662731 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1375/2740] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:05:25 2022] DELETE /identity/v3/users/4761208f3d1a475ab4f3a94b2f5ea2a9 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:25.972055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:25.973903 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:25.989796 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0846f-4350-4420-aefe-6ec192079331 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:26.051729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:26.052554 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:26.052841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:26.053118 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752 admin admin] PATH_INFO: `/v3/projects/886cd133545d4e4e920c3b728e6451ab` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:26.054421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=886cd133545d4e4e920c3b728e6451ab)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:26.056893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:26.127078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '886cd133545d4e4e920c3b728e6451ab', 'request_id': 'req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:26.136808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '886cd133545d4e4e920c3b728e6451ab', 'request_id': 'req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:26.144159 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '886cd133545d4e4e920c3b728e6451ab', 'request_id': 'req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:26.157888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752 admin admin] The token cache is being invalidate because project 886cd133545d4e4e920c3b728e6451ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:05:26.158211 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:26.158921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-4b6d01c6-a86e-4b4e-ada4-3563f81e3752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:26.160548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1366/2741] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:05:25 2022] DELETE /identity/v3/projects/886cd133545d4e4e920c3b728e6451ab => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:26.166914 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919e2b4-1d31-4bab-8456-9f3fc05e8f6a None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:26.167492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919e2b4-1d31-4bab-8456-9f3fc05e8f6a None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:26.167492 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919e2b4-1d31-4bab-8456-9f3fc05e8f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:26.186314 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-0919e2b4-1d31-4bab-8456-9f3fc05e8f6a None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:26.187572 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0919e2b4-1d31-4bab-8456-9f3fc05e8f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:26.251437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1376/2742] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:26.260229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaedcf3-5562-488d-9af4-d432dcd29bf2 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:26.260550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaedcf3-5562-488d-9af4-d432dcd29bf2 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:26.260838 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaedcf3-5562-488d-9af4-d432dcd29bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:26.279803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-ecaedcf3-5562-488d-9af4-d432dcd29bf2 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:26.281065 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ecaedcf3-5562-488d-9af4-d432dcd29bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:26.289546 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1367/2743] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:05:26.293905 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919e2b4-1d31-4bab-8456-9f3fc05e8f6a None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:26.304718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919e2b4-1d31-4bab-8456-9f3fc05e8f6a None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:26.309464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d0dbc-add9-4ba4-a45f-26a6d5b125f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:26.310147 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d0dbc-add9-4ba4-a45f-26a6d5b125f0 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:26.310439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d0dbc-add9-4ba4-a45f-26a6d5b125f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:26.310729 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d0dbc-add9-4ba4-a45f-26a6d5b125f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:26.311467 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d0dbc-add9-4ba4-a45f-26a6d5b125f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:05:26.311778 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d0dbc-add9-4ba4-a45f-26a6d5b125f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:26.313432 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d0dbc-add9-4ba4-a45f-26a6d5b125f0 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:26.326037 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1377/2744] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:05:26.330814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaedcf3-5562-488d-9af4-d432dcd29bf2 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:26.340546 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaedcf3-5562-488d-9af4-d432dcd29bf2 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:26.345821 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c00d4-edbb-4783-af71-092f00a287b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:26.346487 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c00d4-edbb-4783-af71-092f00a287b2 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:26.346771 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c00d4-edbb-4783-af71-092f00a287b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:26.347046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c00d4-edbb-4783-af71-092f00a287b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:26.347754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c00d4-edbb-4783-af71-092f00a287b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:26.349693 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c00d4-edbb-4783-af71-092f00a287b2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:26.379700 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1368/2745] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:05:26 2022] POST /identity/v3/projects => generated 361 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:05:26.380856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d0dbc-add9-4ba4-a45f-26a6d5b125f0 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:26.390995 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d0dbc-add9-4ba4-a45f-26a6d5b125f0 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:26.448354 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e022e-918b-4c7c-a43a-8c18d0f097a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:26.449036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e022e-918b-4c7c-a43a-8c18d0f097a6 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:26.449320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e022e-918b-4c7c-a43a-8c18d0f097a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:26.449597 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e022e-918b-4c7c-a43a-8c18d0f097a6 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:26.450295 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e022e-918b-4c7c-a43a-8c18d0f097a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:26.452236 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79e022e-918b-4c7c-a43a-8c18d0f097a6 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:26.475630 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1378/2746] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:05:26 2022] POST /identity/v3/users => generated 281 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:05:26.477184 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c00d4-edbb-4783-af71-092f00a287b2 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:26.488173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c00d4-edbb-4783-af71-092f00a287b2 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:26.492827 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a2a42-2b32-4fc2-891d-66f95e289feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:26.493495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a2a42-2b32-4fc2-891d-66f95e289feb admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:26.493783 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a2a42-2b32-4fc2-891d-66f95e289feb admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:26.494066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a2a42-2b32-4fc2-891d-66f95e289feb admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:26.494808 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a2a42-2b32-4fc2-891d-66f95e289feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:26.496434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a2a42-2b32-4fc2-891d-66f95e289feb admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:26.507057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1369/2747] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:05:26.511126 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e022e-918b-4c7c-a43a-8c18d0f097a6 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:26.522070 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79e022e-918b-4c7c-a43a-8c18d0f097a6 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:26.526717 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986780a-01b6-4dbf-9531-77af066bcc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:26.527397 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986780a-01b6-4dbf-9531-77af066bcc0b admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:26.527709 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986780a-01b6-4dbf-9531-77af066bcc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:26.528026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986780a-01b6-4dbf-9531-77af066bcc0b admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:26.528774 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986780a-01b6-4dbf-9531-77af066bcc0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:26.530374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c986780a-01b6-4dbf-9531-77af066bcc0b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:26.541489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1379/2748] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:05:26.545762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a2a42-2b32-4fc2-891d-66f95e289feb admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:26.556123 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a2a42-2b32-4fc2-891d-66f95e289feb admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:26.560967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd0bc1-b1dc-49c2-875e-c07e1c282b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:26.561853 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd0bc1-b1dc-49c2-875e-c07e1c282b40 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:26.562132 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd0bc1-b1dc-49c2-875e-c07e1c282b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:26.562403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd0bc1-b1dc-49c2-875e-c07e1c282b40 admin admin] PATH_INFO: `/v3/projects/8f0c5695b09b45e49d2ca880e233cf37/users/c51ae36dca13421292ab20659dda3083/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:26.578563 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd0bc1-b1dc-49c2-875e-c07e1c282b40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f0c5695b09b45e49d2ca880e233cf37, user_id=c51ae36dca13421292ab20659dda3083, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:26.581741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd0bc1-b1dc-49c2-875e-c07e1c282b40 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:26.595350 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1370/2749] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:05:26 2022] PUT /identity/v3/projects/8f0c5695b09b45e49d2ca880e233cf37/users/c51ae36dca13421292ab20659dda3083/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:26.608850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217576be-7734-4390-bce5-3ba2756e4451 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:26.609146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217576be-7734-4390-bce5-3ba2756e4451 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:26.609424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217576be-7734-4390-bce5-3ba2756e4451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:26.636398 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-217576be-7734-4390-bce5-3ba2756e4451 None None] MFA Rules not processed for user `c51ae36dca13421292ab20659dda3083`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:26.637615 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-217576be-7734-4390-bce5-3ba2756e4451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:26.693134 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:26.705127 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1380/2750] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:26.736009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd0bc1-b1dc-49c2-875e-c07e1c282b40 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:26.737919 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b7dd0bc1-b1dc-49c2-875e-c07e1c282b40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:26.751630 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:26.800920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd0bc1-b1dc-49c2-875e-c07e1c282b40 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:26.804603 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd2034-f2b8-46a7-b308-5e58d8df2f22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:26.805331 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2034-f2b8-46a7-b308-5e58d8df2f22 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:26.805620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2034-f2b8-46a7-b308-5e58d8df2f22 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:26.805904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd2034-f2b8-46a7-b308-5e58d8df2f22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:26.815382 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd2034-f2b8-46a7-b308-5e58d8df2f22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:26.816819 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbd2034-f2b8-46a7-b308-5e58d8df2f22 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:26.826502 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1371/2751] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:05:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:05:26.841627 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217576be-7734-4390-bce5-3ba2756e4451 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:26.904946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217576be-7734-4390-bce5-3ba2756e4451 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:26.909622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08034d7-8e30-4af1-b1f7-857c9677466e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:26.910511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08034d7-8e30-4af1-b1f7-857c9677466e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:26.910793 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08034d7-8e30-4af1-b1f7-857c9677466e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:26.911086 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08034d7-8e30-4af1-b1f7-857c9677466e admin admin] PATH_INFO: `/v3/users/c51ae36dca13421292ab20659dda3083` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:26.912320 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08034d7-8e30-4af1-b1f7-857c9677466e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c51ae36dca13421292ab20659dda3083)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:26.914234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08034d7-8e30-4af1-b1f7-857c9677466e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:26.961922 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e08034d7-8e30-4af1-b1f7-857c9677466e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c51ae36dca13421292ab20659dda3083'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:26.973409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e08034d7-8e30-4af1-b1f7-857c9677466e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c51ae36dca13421292ab20659dda3083'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:26.980433 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e08034d7-8e30-4af1-b1f7-857c9677466e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c51ae36dca13421292ab20659dda3083'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:26.981039 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e08034d7-8e30-4af1-b1f7-857c9677466e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c51ae36dca13421292ab20659dda3083'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:26.987948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e08034d7-8e30-4af1-b1f7-857c9677466e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c51ae36dca13421292ab20659dda3083'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:26.993863 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1381/2752] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:05:26 2022] DELETE /identity/v3/users/c51ae36dca13421292ab20659dda3083 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:26.998341 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ec0b2-014b-43c9-ab35-c84ca86e4939 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:26.998984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ec0b2-014b-43c9-ab35-c84ca86e4939 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:26.998984 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ec0b2-014b-43c9-ab35-c84ca86e4939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:27.016208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-c28ec0b2-014b-43c9-ab35-c84ca86e4939 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:27.017900 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c28ec0b2-014b-43c9-ab35-c84ca86e4939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:27.075130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1372/2753] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:05:27.085237 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08034d7-8e30-4af1-b1f7-857c9677466e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:27.086653 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e08034d7-8e30-4af1-b1f7-857c9677466e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:27.144850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08034d7-8e30-4af1-b1f7-857c9677466e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:27.148481 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4858ee9-2ee8-4c3f-aaa4-ce9f34ab2687 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:27.149135 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4858ee9-2ee8-4c3f-aaa4-ce9f34ab2687 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:27.149423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4858ee9-2ee8-4c3f-aaa4-ce9f34ab2687 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:27.149710 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4858ee9-2ee8-4c3f-aaa4-ce9f34ab2687 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:27.158726 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4858ee9-2ee8-4c3f-aaa4-ce9f34ab2687 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:27.160196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4858ee9-2ee8-4c3f-aaa4-ce9f34ab2687 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:27.169688 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1382/2754] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:05:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:05:27.439981 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ec0b2-014b-43c9-ab35-c84ca86e4939 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:27.441411 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c28ec0b2-014b-43c9-ab35-c84ca86e4939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:27.456218 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ec0b2-014b-43c9-ab35-c84ca86e4939 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:27.461836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794f8dcf-9bf5-4ee7-a2b6-531174b69102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:27.462780 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f8dcf-9bf5-4ee7-a2b6-531174b69102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:27.463034 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f8dcf-9bf5-4ee7-a2b6-531174b69102 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:27.463316 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f8dcf-9bf5-4ee7-a2b6-531174b69102 admin admin] PATH_INFO: `/v3/projects/8f0c5695b09b45e49d2ca880e233cf37` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:27.464632 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f8dcf-9bf5-4ee7-a2b6-531174b69102 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f0c5695b09b45e49d2ca880e233cf37)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:27.466747 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794f8dcf-9bf5-4ee7-a2b6-531174b69102 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:27.514137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-794f8dcf-9bf5-4ee7-a2b6-531174b69102 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f0c5695b09b45e49d2ca880e233cf37', 'request_id': 'req-794f8dcf-9bf5-4ee7-a2b6-531174b69102'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:27.521853 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-794f8dcf-9bf5-4ee7-a2b6-531174b69102 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f0c5695b09b45e49d2ca880e233cf37', 'request_id': 'req-794f8dcf-9bf5-4ee7-a2b6-531174b69102'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:27.528156 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-794f8dcf-9bf5-4ee7-a2b6-531174b69102 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f0c5695b09b45e49d2ca880e233cf37', 'request_id': 'req-794f8dcf-9bf5-4ee7-a2b6-531174b69102'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:27.539391 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-794f8dcf-9bf5-4ee7-a2b6-531174b69102 admin admin] The token cache is being invalidate because project 8f0c5695b09b45e49d2ca880e233cf37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:05:27.539680 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-794f8dcf-9bf5-4ee7-a2b6-531174b69102 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:27.540339 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-794f8dcf-9bf5-4ee7-a2b6-531174b69102 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:27.541990 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1373/2755] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:05:27 2022] DELETE /identity/v3/projects/8f0c5695b09b45e49d2ca880e233cf37 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:27.548386 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cebc63-4191-42a0-bce5-886575a9c35b None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:27.548704 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cebc63-4191-42a0-bce5-886575a9c35b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:27.549007 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cebc63-4191-42a0-bce5-886575a9c35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:27.567307 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-d1cebc63-4191-42a0-bce5-886575a9c35b None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:27.568815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d1cebc63-4191-42a0-bce5-886575a9c35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:27.579049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:27.630819 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1383/2756] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:27.693229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0e033-69b5-4427-8798-c3adfd1e6bd7 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:27.693799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0e033-69b5-4427-8798-c3adfd1e6bd7 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:27.694080 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0e033-69b5-4427-8798-c3adfd1e6bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:27.712918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-cac0e033-69b5-4427-8798-c3adfd1e6bd7 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:27.714164 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-cac0e033-69b5-4427-8798-c3adfd1e6bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:27.722651 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1374/2757] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:27.727017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cebc63-4191-42a0-bce5-886575a9c35b None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:27.741144 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cebc63-4191-42a0-bce5-886575a9c35b None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:27.745741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a956a-5f15-4e80-8e47-f78f088b54c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:27.746400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a956a-5f15-4e80-8e47-f78f088b54c1 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:27.746687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a956a-5f15-4e80-8e47-f78f088b54c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:27.746966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a956a-5f15-4e80-8e47-f78f088b54c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:27.747719 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a956a-5f15-4e80-8e47-f78f088b54c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:05:27.748191 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a956a-5f15-4e80-8e47-f78f088b54c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:27.749812 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a956a-5f15-4e80-8e47-f78f088b54c1 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:27.770351 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1384/2758] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:05:27.774787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0e033-69b5-4427-8798-c3adfd1e6bd7 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:27.784762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0e033-69b5-4427-8798-c3adfd1e6bd7 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:27.789414 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5a1ee-f68e-4024-8f85-2e716654cb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:27.790069 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5a1ee-f68e-4024-8f85-2e716654cb6e admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:27.790353 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5a1ee-f68e-4024-8f85-2e716654cb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:27.790630 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f5a1ee-f68e-4024-8f85-2e716654cb6e admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:27.791333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5a1ee-f68e-4024-8f85-2e716654cb6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:27.793256 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f5a1ee-f68e-4024-8f85-2e716654cb6e admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:27.818781 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1375/2759] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:05:27 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:05:27.822906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a956a-5f15-4e80-8e47-f78f088b54c1 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:27.833452 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a956a-5f15-4e80-8e47-f78f088b54c1 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:27.890736 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eace69e-f88c-4bc6-9de7-4bca137b3232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:27.892046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eace69e-f88c-4bc6-9de7-4bca137b3232 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:27.892337 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eace69e-f88c-4bc6-9de7-4bca137b3232 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:27.892619 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eace69e-f88c-4bc6-9de7-4bca137b3232 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:27.893298 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eace69e-f88c-4bc6-9de7-4bca137b3232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:27.895219 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eace69e-f88c-4bc6-9de7-4bca137b3232 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:27.917661 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1385/2760] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:05:27 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:05:27.918769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5a1ee-f68e-4024-8f85-2e716654cb6e admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:27.929683 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f5a1ee-f68e-4024-8f85-2e716654cb6e admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:27.934547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f5bdb-01ee-4118-8ed6-f8fca76c2093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:27.935227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5bdb-01ee-4118-8ed6-f8fca76c2093 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:27.935540 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5bdb-01ee-4118-8ed6-f8fca76c2093 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:27.935817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5bdb-01ee-4118-8ed6-f8fca76c2093 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:27.936583 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f5bdb-01ee-4118-8ed6-f8fca76c2093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:27.938222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f5bdb-01ee-4118-8ed6-f8fca76c2093 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:27.940799 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:27.952323 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1376/2761] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 18:05:27.956016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eace69e-f88c-4bc6-9de7-4bca137b3232 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:27.966256 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eace69e-f88c-4bc6-9de7-4bca137b3232 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:27.971417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88653eec-b597-4f49-959b-4db4f34866e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:27.972126 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88653eec-b597-4f49-959b-4db4f34866e1 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:27.972416 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88653eec-b597-4f49-959b-4db4f34866e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:27.972697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88653eec-b597-4f49-959b-4db4f34866e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:27.973432 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88653eec-b597-4f49-959b-4db4f34866e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:27.975068 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88653eec-b597-4f49-959b-4db4f34866e1 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:27.987200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1386/2762] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 18:05:27.991548 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f5bdb-01ee-4118-8ed6-f8fca76c2093 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:28.001639 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f5bdb-01ee-4118-8ed6-f8fca76c2093 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:28.007023 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc8119-83bb-47de-9a09-5e5cc9a62cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:28.007945 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc8119-83bb-47de-9a09-5e5cc9a62cae admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:28.008242 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc8119-83bb-47de-9a09-5e5cc9a62cae admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:28.008522 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc8119-83bb-47de-9a09-5e5cc9a62cae admin admin] PATH_INFO: `/v3/projects/8739e03d7b5b4284bc3ee4bfb0a86005/users/3a7e212164224f4c8cc76f300ca5f0ca/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:28.025231 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc8119-83bb-47de-9a09-5e5cc9a62cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=8739e03d7b5b4284bc3ee4bfb0a86005, user_id=3a7e212164224f4c8cc76f300ca5f0ca, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:28.027875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc8119-83bb-47de-9a09-5e5cc9a62cae admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:28.041751 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1377/2763] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:05:27 2022] PUT /identity/v3/projects/8739e03d7b5b4284bc3ee4bfb0a86005/users/3a7e212164224f4c8cc76f300ca5f0ca/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:28.052272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5db2d-ba0e-4fb7-be06-2193e5529733 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:28.052564 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5db2d-ba0e-4fb7-be06-2193e5529733 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:28.052981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb5db2d-ba0e-4fb7-be06-2193e5529733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:28.088490 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-4eb5db2d-ba0e-4fb7-be06-2193e5529733 None None] MFA Rules not processed for user `3a7e212164224f4c8cc76f300ca5f0ca`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:28.089658 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4eb5db2d-ba0e-4fb7-be06-2193e5529733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:28.154106 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1387/2764] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:05:28.165388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc8119-83bb-47de-9a09-5e5cc9a62cae admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:28.167062 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f0bc8119-83bb-47de-9a09-5e5cc9a62cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:28.222441 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc8119-83bb-47de-9a09-5e5cc9a62cae admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:28.226002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d84e45-e3f1-4649-9d69-218e03808260 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:28.226641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d84e45-e3f1-4649-9d69-218e03808260 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:28.226921 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d84e45-e3f1-4649-9d69-218e03808260 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:28.227193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d84e45-e3f1-4649-9d69-218e03808260 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:28.235270 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d84e45-e3f1-4649-9d69-218e03808260 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:28.236712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d84e45-e3f1-4649-9d69-218e03808260 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:28.245325 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1378/2765] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:05:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:05:38.142109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5db2d-ba0e-4fb7-be06-2193e5529733 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:38.157944 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:38.215723 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb5db2d-ba0e-4fb7-be06-2193e5529733 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:38.220429 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e73ae-17a8-4eaf-b7c1-4a95d7d09612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:38.221328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e73ae-17a8-4eaf-b7c1-4a95d7d09612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:38.221606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e73ae-17a8-4eaf-b7c1-4a95d7d09612 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:38.221885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8e73ae-17a8-4eaf-b7c1-4a95d7d09612 admin admin] PATH_INFO: `/v3/users/3a7e212164224f4c8cc76f300ca5f0ca` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:38.223109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e73ae-17a8-4eaf-b7c1-4a95d7d09612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a7e212164224f4c8cc76f300ca5f0ca)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:38.225074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8e73ae-17a8-4eaf-b7c1-4a95d7d09612 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:38.242312 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:38.277242 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-cc8e73ae-17a8-4eaf-b7c1-4a95d7d09612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a7e212164224f4c8cc76f300ca5f0ca'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:38.289411 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-cc8e73ae-17a8-4eaf-b7c1-4a95d7d09612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a7e212164224f4c8cc76f300ca5f0ca'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:38.297629 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-cc8e73ae-17a8-4eaf-b7c1-4a95d7d09612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a7e212164224f4c8cc76f300ca5f0ca'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:38.298278 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-cc8e73ae-17a8-4eaf-b7c1-4a95d7d09612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a7e212164224f4c8cc76f300ca5f0ca'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:38.309674 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-cc8e73ae-17a8-4eaf-b7c1-4a95d7d09612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a7e212164224f4c8cc76f300ca5f0ca'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:38.316905 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1388/2766] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:05:38 2022] DELETE /identity/v3/users/3a7e212164224f4c8cc76f300ca5f0ca => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:38.321516 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ef54f-2126-48b7-ba1e-3aaad5d30456 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:38.321836 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ef54f-2126-48b7-ba1e-3aaad5d30456 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:38.322113 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ef54f-2126-48b7-ba1e-3aaad5d30456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:38.340064 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4a4ef54f-2126-48b7-ba1e-3aaad5d30456 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:05:38.341271 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4a4ef54f-2126-48b7-ba1e-3aaad5d30456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:38.409860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1379/2767] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 18:05:38.420239 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e73ae-17a8-4eaf-b7c1-4a95d7d09612 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:38.421940 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-cc8e73ae-17a8-4eaf-b7c1-4a95d7d09612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:38.481986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8e73ae-17a8-4eaf-b7c1-4a95d7d09612 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:38.486251 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6160d-00b2-41c2-9cef-47b7d86c6d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:38.486912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6160d-00b2-41c2-9cef-47b7d86c6d1a service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:38.487197 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6160d-00b2-41c2-9cef-47b7d86c6d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:38.487476 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6160d-00b2-41c2-9cef-47b7d86c6d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:38.497358 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6160d-00b2-41c2-9cef-47b7d86c6d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:38.499504 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e6160d-00b2-41c2-9cef-47b7d86c6d1a service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:38.509803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1389/2768] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:05:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 18:05:38.801880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ef54f-2126-48b7-ba1e-3aaad5d30456 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:05:38.803515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4a4ef54f-2126-48b7-ba1e-3aaad5d30456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:05:38.816367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ef54f-2126-48b7-ba1e-3aaad5d30456 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:05:38.821580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:05:38.822457 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:05:38.822769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:05:38.823055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] PATH_INFO: `/v3/projects/8739e03d7b5b4284bc3ee4bfb0a86005` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:05:38.824376 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8739e03d7b5b4284bc3ee4bfb0a86005)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:05:38.826365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:05:38.879216 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8739e03d7b5b4284bc3ee4bfb0a86005', 'request_id': 'req-28ea711a-224e-4083-9d85-2bc18e380f0c'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:38.887158 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8739e03d7b5b4284bc3ee4bfb0a86005', 'request_id': 'req-28ea711a-224e-4083-9d85-2bc18e380f0c'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:38.894698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8739e03d7b5b4284bc3ee4bfb0a86005', 'request_id': 'req-28ea711a-224e-4083-9d85-2bc18e380f0c'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:38.907082 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] The token cache is being invalidate because project 8739e03d7b5b4284bc3ee4bfb0a86005 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:05:38.907499 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:38.908202 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:05:38.909754 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1380/2769] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:05:38 2022] DELETE /identity/v3/projects/8739e03d7b5b4284bc3ee4bfb0a86005 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:05:48.505248 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=74,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:05:48.904023 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:06:01.425668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2aef3b-efa1-4f76-b911-14870385c35b None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:06:01.426312 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2aef3b-efa1-4f76-b911-14870385c35b None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:06:01.426312 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2aef3b-efa1-4f76-b911-14870385c35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:06:01.445540 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-dd2aef3b-efa1-4f76-b911-14870385c35b None None] MFA Rules not processed for user `d1a7b2376e83489b87feb4b9fd449d55`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:06:01.446912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-dd2aef3b-efa1-4f76-b911-14870385c35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:06:01.515078 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1390/2770] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:06:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:06:02.216866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:06:02.218498 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:06:02.349600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ea711a-224e-4083-9d85-2bc18e380f0c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:06:02.357583 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b966a70-e60c-40ae-871e-2adb5cb80730 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:06:02.358657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b966a70-e60c-40ae-871e-2adb5cb80730 service placement] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:06:02.359129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b966a70-e60c-40ae-871e-2adb5cb80730 service placement] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:06:02.359604 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b966a70-e60c-40ae-871e-2adb5cb80730 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:06:02.362654 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9b966a70-e60c-40ae-871e-2adb5cb80730 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:06:02.383474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b966a70-e60c-40ae-871e-2adb5cb80730 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:06:02.386622 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b966a70-e60c-40ae-871e-2adb5cb80730 service placement] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:06:02.405038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1381/2771] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:06:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:06:11.507101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=25 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:06:12.397003 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=32 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:06:15.749462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2aef3b-efa1-4f76-b911-14870385c35b None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:06:15.750958 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-dd2aef3b-efa1-4f76-b911-14870385c35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:06:15.833715 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2aef3b-efa1-4f76-b911-14870385c35b None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:06:15.840005 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d623f3-a336-429f-b595-0c9a4f9ee51b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:06:15.840833 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d623f3-a336-429f-b595-0c9a4f9ee51b service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:06:15.841122 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d623f3-a336-429f-b595-0c9a4f9ee51b service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:06:15.841403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d623f3-a336-429f-b595-0c9a4f9ee51b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:06:15.843441 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-14d623f3-a336-429f-b595-0c9a4f9ee51b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:06:15.859509 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d623f3-a336-429f-b595-0c9a4f9ee51b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:06:15.861196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d623f3-a336-429f-b595-0c9a4f9ee51b service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:06:15.873243 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1391/2772] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:06:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:06:25.867969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=34 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:07:31.194139 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b966a70-e60c-40ae-871e-2adb5cb80730 service placement] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:07:31.195562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9b966a70-e60c-40ae-871e-2adb5cb80730 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:07:31.251480 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b966a70-e60c-40ae-871e-2adb5cb80730 service placement] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:07:31.255142 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbb7673-435c-421c-b355-8b391e6d2e9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:07:31.255786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb7673-435c-421c-b355-8b391e6d2e9a service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:07:31.256103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb7673-435c-421c-b355-8b391e6d2e9a service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:07:31.256387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbb7673-435c-421c-b355-8b391e6d2e9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:07:31.258199 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-cbbb7673-435c-421c-b355-8b391e6d2e9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:07:31.272332 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb7673-435c-421c-b355-8b391e6d2e9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:07:31.273776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbb7673-435c-421c-b355-8b391e6d2e9a service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:07:31.284656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1382/2773] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:07:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:07:32.684291 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d623f3-a336-429f-b595-0c9a4f9ee51b service nova] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:07:32.685732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-14d623f3-a336-429f-b595-0c9a4f9ee51b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:07:32.740745 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d623f3-a336-429f-b595-0c9a4f9ee51b service nova] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:07:32.744813 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10421d39-fd25-4fe3-baea-32528a09859d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:07:32.745485 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10421d39-fd25-4fe3-baea-32528a09859d service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:07:32.745769 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10421d39-fd25-4fe3-baea-32528a09859d service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:07:32.746049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10421d39-fd25-4fe3-baea-32528a09859d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:07:32.747996 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-10421d39-fd25-4fe3-baea-32528a09859d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:07:32.761955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10421d39-fd25-4fe3-baea-32528a09859d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:07:32.763374 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10421d39-fd25-4fe3-baea-32528a09859d service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:07:32.774319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1392/2774] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:07:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:07:41.279762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=26 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:07:42.630135 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b9885-88a2-4367-afdf-da3ee266495d None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:07:42.630814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b9885-88a2-4367-afdf-da3ee266495d None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:07:42.630814 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b9885-88a2-4367-afdf-da3ee266495d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:07:42.647663 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4d7b9885-88a2-4367-afdf-da3ee266495d None None] MFA Rules not processed for user `d1a7b2376e83489b87feb4b9fd449d55`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:07:42.648904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4d7b9885-88a2-4367-afdf-da3ee266495d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:07:42.655617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1383/2775] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:07:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:07:42.769377 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=26 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:07:45.004440 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10421d39-fd25-4fe3-baea-32528a09859d service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:07:45.005968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-10421d39-fd25-4fe3-baea-32528a09859d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:07:45.017550 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10421d39-fd25-4fe3-baea-32528a09859d service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:07:45.021383 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14848bca-92eb-48ac-8fdb-3d1eadacc42d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:07:45.022122 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14848bca-92eb-48ac-8fdb-3d1eadacc42d service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:07:45.022521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14848bca-92eb-48ac-8fdb-3d1eadacc42d service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:07:45.022875 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14848bca-92eb-48ac-8fdb-3d1eadacc42d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:07:45.024953 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-14848bca-92eb-48ac-8fdb-3d1eadacc42d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:07:45.035742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14848bca-92eb-48ac-8fdb-3d1eadacc42d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:07:45.037214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14848bca-92eb-48ac-8fdb-3d1eadacc42d service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:07:45.046015 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1393/2776] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:07:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:07:52.642936 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:07:55.042279 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=6 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:09:18.837957 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b9885-88a2-4367-afdf-da3ee266495d None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:09:18.884571 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b9885-88a2-4367-afdf-da3ee266495d None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:09:18.890155 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4246d97-70f2-40e3-96fc-cef49dbbabab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:09:18.890816 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4246d97-70f2-40e3-96fc-cef49dbbabab service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:09:18.891100 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4246d97-70f2-40e3-96fc-cef49dbbabab service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:09:18.891380 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4246d97-70f2-40e3-96fc-cef49dbbabab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:09:18.893267 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-b4246d97-70f2-40e3-96fc-cef49dbbabab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:09:18.913967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4246d97-70f2-40e3-96fc-cef49dbbabab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:09:18.915541 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4246d97-70f2-40e3-96fc-cef49dbbabab service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:09:18.925274 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1384/2777] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:09:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:09:28.920928 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=24 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:09:35.421744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810d4e6-d20f-442f-a732-5db4d12f4739 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:09:35.422301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810d4e6-d20f-442f-a732-5db4d12f4739 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:09:35.422301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810d4e6-d20f-442f-a732-5db4d12f4739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:09:35.439105 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-1810d4e6-d20f-442f-a732-5db4d12f4739 None None] MFA Rules not processed for user `d1a7b2376e83489b87feb4b9fd449d55`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:09:35.440344 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1810d4e6-d20f-442f-a732-5db4d12f4739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:09:35.447670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1394/2778] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:09:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:09:35.741468 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4246d97-70f2-40e3-96fc-cef49dbbabab service nova] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:09:35.751021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4246d97-70f2-40e3-96fc-cef49dbbabab service nova] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:09:35.755241 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984f882-aac5-4bda-92c1-a4e9e84515c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:09:35.755883 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984f882-aac5-4bda-92c1-a4e9e84515c0 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:09:35.756208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984f882-aac5-4bda-92c1-a4e9e84515c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:09:35.756489 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4984f882-aac5-4bda-92c1-a4e9e84515c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:09:35.758287 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4984f882-aac5-4bda-92c1-a4e9e84515c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:09:35.778841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984f882-aac5-4bda-92c1-a4e9e84515c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:09:35.780305 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4984f882-aac5-4bda-92c1-a4e9e84515c0 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:09:35.788284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1385/2779] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:09:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:09:36.133140 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810d4e6-d20f-442f-a732-5db4d12f4739 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:09:36.134559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1810d4e6-d20f-442f-a732-5db4d12f4739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:09:36.152420 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810d4e6-d20f-442f-a732-5db4d12f4739 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:09:36.159527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4640f-8b6d-4e9b-b739-05c970ac0c7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:09:36.160667 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4640f-8b6d-4e9b-b739-05c970ac0c7d service placement] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:09:36.161164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4640f-8b6d-4e9b-b739-05c970ac0c7d service placement] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:09:36.161602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4640f-8b6d-4e9b-b739-05c970ac0c7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:09:36.164447 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-77c4640f-8b6d-4e9b-b739-05c970ac0c7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:09:36.180625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4640f-8b6d-4e9b-b739-05c970ac0c7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:09:36.182253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c4640f-8b6d-4e9b-b739-05c970ac0c7d service placement] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:09:36.193538 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1395/2780] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:09:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:09:44.212698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984f882-aac5-4bda-92c1-a4e9e84515c0 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:09:44.229565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4984f882-aac5-4bda-92c1-a4e9e84515c0 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:09:44.235377 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05405d35-28a4-4c35-b9a5-ce8b6f6342bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:09:44.236072 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05405d35-28a4-4c35-b9a5-ce8b6f6342bb service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:09:44.236365 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05405d35-28a4-4c35-b9a5-ce8b6f6342bb service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:09:44.236646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05405d35-28a4-4c35-b9a5-ce8b6f6342bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:09:44.238529 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-05405d35-28a4-4c35-b9a5-ce8b6f6342bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:09:44.251306 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05405d35-28a4-4c35-b9a5-ce8b6f6342bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:09:44.252974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05405d35-28a4-4c35-b9a5-ce8b6f6342bb service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:09:44.266832 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1386/2781] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:09:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:09:46.188434 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=9 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:09:54.261054 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=16 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:07.149582 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4640f-8b6d-4e9b-b739-05c970ac0c7d service placement] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:07.171850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4640f-8b6d-4e9b-b739-05c970ac0c7d service placement] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:07.177718 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1a2f0-2926-44de-a150-7d70c2556ddb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:07.178410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1a2f0-2926-44de-a150-7d70c2556ddb service placement] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:07.178755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1a2f0-2926-44de-a150-7d70c2556ddb service placement] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:07.178981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f1a2f0-2926-44de-a150-7d70c2556ddb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:07.189655 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1a2f0-2926-44de-a150-7d70c2556ddb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:07.191271 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f1a2f0-2926-44de-a150-7d70c2556ddb service placement] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:07.202677 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1396/2782] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:11:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:11:17.197721 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=10 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:25.872367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05405d35-28a4-4c35-b9a5-ce8b6f6342bb service nova] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:25.873898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-05405d35-28a4-4c35-b9a5-ce8b6f6342bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:25.929623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05405d35-28a4-4c35-b9a5-ce8b6f6342bb service nova] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:25.937227 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f5418-f196-4bcd-b1e4-56b518495642 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:25.938511 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f5418-f196-4bcd-b1e4-56b518495642 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:25.939191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f5418-f196-4bcd-b1e4-56b518495642 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:25.939707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f5418-f196-4bcd-b1e4-56b518495642 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:25.941958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-255f5418-f196-4bcd-b1e4-56b518495642 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:25.957844 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f5418-f196-4bcd-b1e4-56b518495642 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:25.959831 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f5418-f196-4bcd-b1e4-56b518495642 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:25.976220 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1387/2783] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:11:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:11:31.053186 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1a2f0-2926-44de-a150-7d70c2556ddb service placement] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:31.072481 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f1a2f0-2926-44de-a150-7d70c2556ddb service placement] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:31.076074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccce11-1fa5-427a-9103-abf68b6acf4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:31.076715 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccce11-1fa5-427a-9103-abf68b6acf4f service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:31.077003 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccce11-1fa5-427a-9103-abf68b6acf4f service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:31.077284 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccce11-1fa5-427a-9103-abf68b6acf4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:31.079046 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b8ccce11-1fa5-427a-9103-abf68b6acf4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:31.163825 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccce11-1fa5-427a-9103-abf68b6acf4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:31.165304 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccce11-1fa5-427a-9103-abf68b6acf4f service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:31.184157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1397/2784] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:11:34.186150 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f5418-f196-4bcd-b1e4-56b518495642 service nova] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:34.198933 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f5418-f196-4bcd-b1e4-56b518495642 service nova] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:34.204316 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b8940-5c13-438c-8fd5-cd8888abf394 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:34.205136 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b8940-5c13-438c-8fd5-cd8888abf394 service placement] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:34.205527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b8940-5c13-438c-8fd5-cd8888abf394 service placement] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:34.205908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b8940-5c13-438c-8fd5-cd8888abf394 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:34.207846 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-fd7b8940-5c13-438c-8fd5-cd8888abf394 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:34.237928 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b8940-5c13-438c-8fd5-cd8888abf394 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:34.239715 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b8940-5c13-438c-8fd5-cd8888abf394 service placement] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:34.255324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccce11-1fa5-427a-9103-abf68b6acf4f service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:34.257841 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b8ccce11-1fa5-427a-9103-abf68b6acf4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:34.265528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1388/2785] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:11:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:11:34.274855 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccce11-1fa5-427a-9103-abf68b6acf4f service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:34.280145 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a654c5-135d-4591-8078-ed96de2a114a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:34.281572 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a654c5-135d-4591-8078-ed96de2a114a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:34.281972 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a654c5-135d-4591-8078-ed96de2a114a admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:34.282803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a654c5-135d-4591-8078-ed96de2a114a admin admin] PATH_INFO: `/v3/users/d1a7b2376e83489b87feb4b9fd449d55` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:34.284306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a654c5-135d-4591-8078-ed96de2a114a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1a7b2376e83489b87feb4b9fd449d55)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:34.286526 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a654c5-135d-4591-8078-ed96de2a114a admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:34.336052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-39a654c5-135d-4591-8078-ed96de2a114a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1a7b2376e83489b87feb4b9fd449d55'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:34.348306 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-39a654c5-135d-4591-8078-ed96de2a114a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1a7b2376e83489b87feb4b9fd449d55'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:34.355541 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-39a654c5-135d-4591-8078-ed96de2a114a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1a7b2376e83489b87feb4b9fd449d55'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:34.356195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-39a654c5-135d-4591-8078-ed96de2a114a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1a7b2376e83489b87feb4b9fd449d55'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:34.363658 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-39a654c5-135d-4591-8078-ed96de2a114a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1a7b2376e83489b87feb4b9fd449d55'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:34.369528 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1398/2786] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:11:34 2022] DELETE /identity/v3/users/d1a7b2376e83489b87feb4b9fd449d55 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:34.547180 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b8940-5c13-438c-8fd5-cd8888abf394 service placement] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:34.560222 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b8940-5c13-438c-8fd5-cd8888abf394 service placement] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:34.618300 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aee507-7c2b-42db-b196-6064b21fb42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:34.619147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aee507-7c2b-42db-b196-6064b21fb42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:34.619460 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aee507-7c2b-42db-b196-6064b21fb42b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:34.619743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aee507-7c2b-42db-b196-6064b21fb42b admin admin] PATH_INFO: `/v3/projects/cd7b0d70fffe4d37a84bd4ee73d044ca` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:34.620967 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aee507-7c2b-42db-b196-6064b21fb42b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd7b0d70fffe4d37a84bd4ee73d044ca)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:34.622900 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aee507-7c2b-42db-b196-6064b21fb42b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:34.668985 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d8aee507-7c2b-42db-b196-6064b21fb42b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd7b0d70fffe4d37a84bd4ee73d044ca', 'request_id': 'req-d8aee507-7c2b-42db-b196-6064b21fb42b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:34.676111 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d8aee507-7c2b-42db-b196-6064b21fb42b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd7b0d70fffe4d37a84bd4ee73d044ca', 'request_id': 'req-d8aee507-7c2b-42db-b196-6064b21fb42b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:34.681645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d8aee507-7c2b-42db-b196-6064b21fb42b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd7b0d70fffe4d37a84bd4ee73d044ca', 'request_id': 'req-d8aee507-7c2b-42db-b196-6064b21fb42b'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:34.692910 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d8aee507-7c2b-42db-b196-6064b21fb42b admin admin] The token cache is being invalidate because project cd7b0d70fffe4d37a84bd4ee73d044ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:11:34.693200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d8aee507-7c2b-42db-b196-6064b21fb42b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:34.693832 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d8aee507-7c2b-42db-b196-6064b21fb42b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:34.695344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1389/2787] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:11:34 2022] DELETE /identity/v3/projects/cd7b0d70fffe4d37a84bd4ee73d044ca => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:34.700477 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1963b-9015-4789-9dd2-e7f5114d8e6f None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:34.700770 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1963b-9015-4789-9dd2-e7f5114d8e6f None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:34.701055 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e1963b-9015-4789-9dd2-e7f5114d8e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:34.738599 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:34.759790 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-91e1963b-9015-4789-9dd2-e7f5114d8e6f None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:11:34.760969 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-91e1963b-9015-4789-9dd2-e7f5114d8e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:34.819525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1399/2788] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:11:34.827430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89eb6b-4868-494f-bbce-f58009e64fdc None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:34.827717 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89eb6b-4868-494f-bbce-f58009e64fdc None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:34.828044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b89eb6b-4868-494f-bbce-f58009e64fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:34.847874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4b89eb6b-4868-494f-bbce-f58009e64fdc None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:11:34.849130 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4b89eb6b-4868-494f-bbce-f58009e64fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:34.857175 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1390/2789] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:11:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:11:34.862142 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1963b-9015-4789-9dd2-e7f5114d8e6f None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:34.871813 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e1963b-9015-4789-9dd2-e7f5114d8e6f None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:34.876381 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2563b-3821-455b-bc46-1139f059a30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:34.877036 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2563b-3821-455b-bc46-1139f059a30f admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:34.877319 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2563b-3821-455b-bc46-1139f059a30f admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:34.877596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f2563b-3821-455b-bc46-1139f059a30f admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:34.878323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2563b-3821-455b-bc46-1139f059a30f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:11:34.878631 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2563b-3821-455b-bc46-1139f059a30f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:34.880258 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f2563b-3821-455b-bc46-1139f059a30f admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:34.892188 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1400/2790] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:11:34.896383 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89eb6b-4868-494f-bbce-f58009e64fdc None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:34.905472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b89eb6b-4868-494f-bbce-f58009e64fdc None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:34.910875 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f2cfe-7ccb-4295-81ac-2c78ef565a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:34.911573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f2cfe-7ccb-4295-81ac-2c78ef565a5b admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:34.911860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f2cfe-7ccb-4295-81ac-2c78ef565a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:34.912198 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f2cfe-7ccb-4295-81ac-2c78ef565a5b admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:34.912908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f2cfe-7ccb-4295-81ac-2c78ef565a5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:34.914834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f2cfe-7ccb-4295-81ac-2c78ef565a5b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:34.932389 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:34.950847 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1391/2791] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:11:34 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:11:34.955465 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2563b-3821-455b-bc46-1139f059a30f admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:34.965824 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f2563b-3821-455b-bc46-1139f059a30f admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:35.022152 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822eab6-efe6-4588-80f0-41ecb273577b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:35.022745 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822eab6-efe6-4588-80f0-41ecb273577b admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:35.023031 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822eab6-efe6-4588-80f0-41ecb273577b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:35.023312 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8822eab6-efe6-4588-80f0-41ecb273577b admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:35.024039 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822eab6-efe6-4588-80f0-41ecb273577b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:35.025946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8822eab6-efe6-4588-80f0-41ecb273577b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:35.045617 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1401/2792] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:11:34 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:11:35.049703 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f2cfe-7ccb-4295-81ac-2c78ef565a5b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:35.059817 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f2cfe-7ccb-4295-81ac-2c78ef565a5b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:35.064810 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8960023-2bc2-4173-9920-8c1f9af85bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:35.065509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8960023-2bc2-4173-9920-8c1f9af85bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:35.065822 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8960023-2bc2-4173-9920-8c1f9af85bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:35.066125 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8960023-2bc2-4173-9920-8c1f9af85bb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:35.066860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8960023-2bc2-4173-9920-8c1f9af85bb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:35.068505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8960023-2bc2-4173-9920-8c1f9af85bb2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:35.078528 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1392/2793] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:11:35.082822 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822eab6-efe6-4588-80f0-41ecb273577b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:35.092848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8822eab6-efe6-4588-80f0-41ecb273577b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:35.098107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74030d7c-bc96-4ea7-837c-5b2f366ef446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:35.098785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74030d7c-bc96-4ea7-837c-5b2f366ef446 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:35.099076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74030d7c-bc96-4ea7-837c-5b2f366ef446 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:35.099360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74030d7c-bc96-4ea7-837c-5b2f366ef446 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:35.100125 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74030d7c-bc96-4ea7-837c-5b2f366ef446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:35.101712 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74030d7c-bc96-4ea7-837c-5b2f366ef446 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:35.111946 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1402/2794] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:11:35.116193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8960023-2bc2-4173-9920-8c1f9af85bb2 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:35.126485 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8960023-2bc2-4173-9920-8c1f9af85bb2 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:35.131129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc002d-6105-495d-8662-82e90b4ca259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:35.132119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc002d-6105-495d-8662-82e90b4ca259 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:35.132406 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc002d-6105-495d-8662-82e90b4ca259 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:35.132688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc002d-6105-495d-8662-82e90b4ca259 admin admin] PATH_INFO: `/v3/projects/5bdc1c06b6894c928c3b2595cfa90ff8/users/81ed691f4b924d7f931c0244b275342c/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:35.147696 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc002d-6105-495d-8662-82e90b4ca259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bdc1c06b6894c928c3b2595cfa90ff8, user_id=81ed691f4b924d7f931c0244b275342c, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:35.150564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc002d-6105-495d-8662-82e90b4ca259 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:35.164346 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1393/2795] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:11:35 2022] PUT /identity/v3/projects/5bdc1c06b6894c928c3b2595cfa90ff8/users/81ed691f4b924d7f931c0244b275342c/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:35.174214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f52aa-95dd-4451-9b87-74427dc04dfb None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:35.174497 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f52aa-95dd-4451-9b87-74427dc04dfb None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:35.174783 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f52aa-95dd-4451-9b87-74427dc04dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:35.200542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-334f52aa-95dd-4451-9b87-74427dc04dfb None None] MFA Rules not processed for user `81ed691f4b924d7f931c0244b275342c`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:11:35.201758 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-334f52aa-95dd-4451-9b87-74427dc04dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:35.257551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:35.269026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1403/2796] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:11:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:11:35.273734 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc002d-6105-495d-8662-82e90b4ca259 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:35.334229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc002d-6105-495d-8662-82e90b4ca259 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:35.338925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba39ed-8540-4c82-81c7-9b95d4734a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:35.339585 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba39ed-8540-4c82-81c7-9b95d4734a9c admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:35.339872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba39ed-8540-4c82-81c7-9b95d4734a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:35.340193 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba39ed-8540-4c82-81c7-9b95d4734a9c admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:35.340883 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba39ed-8540-4c82-81c7-9b95d4734a9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:35.342769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba39ed-8540-4c82-81c7-9b95d4734a9c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:35.368815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1394/2797] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:11:35 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:11:35.372988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f52aa-95dd-4451-9b87-74427dc04dfb None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:35.383858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f52aa-95dd-4451-9b87-74427dc04dfb None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:35.441153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca933-ecc7-4362-b249-ba79e78122a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:35.441864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca933-ecc7-4362-b249-ba79e78122a0 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:35.442131 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca933-ecc7-4362-b249-ba79e78122a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:35.442410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca933-ecc7-4362-b249-ba79e78122a0 admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:35.443204 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aca933-ecc7-4362-b249-ba79e78122a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:35.445151 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aca933-ecc7-4362-b249-ba79e78122a0 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:35.465403 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1404/2798] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:11:35 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:11:35.469206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba39ed-8540-4c82-81c7-9b95d4734a9c admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:35.480368 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba39ed-8540-4c82-81c7-9b95d4734a9c admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:35.484983 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0a21e-0618-48ff-a6c5-6093259e05e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:35.486182 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0a21e-0618-48ff-a6c5-6093259e05e8 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:35.486472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0a21e-0618-48ff-a6c5-6093259e05e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:35.486753 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf0a21e-0618-48ff-a6c5-6093259e05e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:35.487480 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0a21e-0618-48ff-a6c5-6093259e05e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:35.489108 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf0a21e-0618-48ff-a6c5-6093259e05e8 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:35.499775 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1395/2799] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:11:35.503897 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca933-ecc7-4362-b249-ba79e78122a0 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:35.514124 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca933-ecc7-4362-b249-ba79e78122a0 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:35.519196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c65c0b-c7cd-44d3-86ec-55d688ef27b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:35.519838 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c65c0b-c7cd-44d3-86ec-55d688ef27b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:35.520156 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c65c0b-c7cd-44d3-86ec-55d688ef27b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:35.520437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c65c0b-c7cd-44d3-86ec-55d688ef27b1 admin admin] PATH_INFO: `/v3/projects/b20d336da9ff42d8972903dcc9d01e0a/users/70af958310044bd291fe46651e405f29/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:35.537553 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c65c0b-c7cd-44d3-86ec-55d688ef27b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b20d336da9ff42d8972903dcc9d01e0a, user_id=70af958310044bd291fe46651e405f29, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:35.540495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c65c0b-c7cd-44d3-86ec-55d688ef27b1 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:35.557405 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1405/2800] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:11:35 2022] PUT /identity/v3/projects/b20d336da9ff42d8972903dcc9d01e0a/users/70af958310044bd291fe46651e405f29/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:35.561446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0a21e-0618-48ff-a6c5-6093259e05e8 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:35.572253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf0a21e-0618-48ff-a6c5-6093259e05e8 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:35.633187 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588cdf2-2853-4da7-9a46-151fd9115f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:35.633858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588cdf2-2853-4da7-9a46-151fd9115f89 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:35.634144 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588cdf2-2853-4da7-9a46-151fd9115f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:35.634423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588cdf2-2853-4da7-9a46-151fd9115f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:35.635154 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588cdf2-2853-4da7-9a46-151fd9115f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:35.636786 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588cdf2-2853-4da7-9a46-151fd9115f89 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:35.647329 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1396/2801] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:11:35.651511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c65c0b-c7cd-44d3-86ec-55d688ef27b1 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:35.662436 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c65c0b-c7cd-44d3-86ec-55d688ef27b1 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:35.667143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3f7a7-5603-422e-a4db-148790d47a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:35.668059 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3f7a7-5603-422e-a4db-148790d47a18 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:35.668346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3f7a7-5603-422e-a4db-148790d47a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:35.668622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3f7a7-5603-422e-a4db-148790d47a18 admin admin] PATH_INFO: `/v3/projects/b20d336da9ff42d8972903dcc9d01e0a/users/70af958310044bd291fe46651e405f29/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:35.670612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3f7a7-5603-422e-a4db-148790d47a18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b20d336da9ff42d8972903dcc9d01e0a, user_id=70af958310044bd291fe46651e405f29, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:35.673310 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3f7a7-5603-422e-a4db-148790d47a18 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:35.683479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1406/2802] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:11:35 2022] PUT /identity/v3/projects/b20d336da9ff42d8972903dcc9d01e0a/users/70af958310044bd291fe46651e405f29/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:35.694731 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5e26d-a084-4392-98ff-7baceee836e0 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:35.695054 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5e26d-a084-4392-98ff-7baceee836e0 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:35.695367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5e26d-a084-4392-98ff-7baceee836e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:35.722505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-bbd5e26d-a084-4392-98ff-7baceee836e0 None None] MFA Rules not processed for user `70af958310044bd291fe46651e405f29`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:11:35.723710 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-bbd5e26d-a084-4392-98ff-7baceee836e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:35.735455 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:35.795230 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1397/2803] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:11:35 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:11:35.806700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3f7a7-5603-422e-a4db-148790d47a18 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:35.808413 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-33c3f7a7-5603-422e-a4db-148790d47a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:35.889099 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3f7a7-5603-422e-a4db-148790d47a18 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:35.893759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d02e5-b831-4813-8109-091d915fee0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:35.894414 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d02e5-b831-4813-8109-091d915fee0b service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:35.894747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d02e5-b831-4813-8109-091d915fee0b service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:35.895032 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d02e5-b831-4813-8109-091d915fee0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:35.904266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d02e5-b831-4813-8109-091d915fee0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:35.905707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7d02e5-b831-4813-8109-091d915fee0b service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:35.915662 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1407/2804] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:11:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:11:35.984573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5e26d-a084-4392-98ff-7baceee836e0 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:35.994078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5e26d-a084-4392-98ff-7baceee836e0 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:35.997655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd095b0d-cddb-4844-bde2-61a5d451c92d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:35.998295 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd095b0d-cddb-4844-bde2-61a5d451c92d service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:35.998578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd095b0d-cddb-4844-bde2-61a5d451c92d service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:35.998854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd095b0d-cddb-4844-bde2-61a5d451c92d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:36.054334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd095b0d-cddb-4844-bde2-61a5d451c92d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:36.055692 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd095b0d-cddb-4844-bde2-61a5d451c92d service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:36.065048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1398/2805] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:11:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:11:36.326882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d02e5-b831-4813-8109-091d915fee0b service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:36.357755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:36.390756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7d02e5-b831-4813-8109-091d915fee0b service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:36.399608 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a0fe9-b348-4578-8683-49ece6caaf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:36.400530 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a0fe9-b348-4578-8683-49ece6caaf64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:36.400815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a0fe9-b348-4578-8683-49ece6caaf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:36.401091 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a0fe9-b348-4578-8683-49ece6caaf64 admin admin] PATH_INFO: `/v3/users/81ed691f4b924d7f931c0244b275342c` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:36.402276 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a0fe9-b348-4578-8683-49ece6caaf64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ed691f4b924d7f931c0244b275342c)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:36.404234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a0fe9-b348-4578-8683-49ece6caaf64 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:36.452197 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-657a0fe9-b348-4578-8683-49ece6caaf64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ed691f4b924d7f931c0244b275342c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:36.463990 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-657a0fe9-b348-4578-8683-49ece6caaf64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ed691f4b924d7f931c0244b275342c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:36.471101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-657a0fe9-b348-4578-8683-49ece6caaf64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ed691f4b924d7f931c0244b275342c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:36.471771 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-657a0fe9-b348-4578-8683-49ece6caaf64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ed691f4b924d7f931c0244b275342c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:36.478760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-657a0fe9-b348-4578-8683-49ece6caaf64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ed691f4b924d7f931c0244b275342c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:36.484700 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1408/2806] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:11:36 2022] DELETE /identity/v3/users/81ed691f4b924d7f931c0244b275342c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:36.489719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1fa85-ddcb-4649-9a1e-78e353911027 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:36.490206 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1fa85-ddcb-4649-9a1e-78e353911027 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:36.490336 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1fa85-ddcb-4649-9a1e-78e353911027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:36.507732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-f4b1fa85-ddcb-4649-9a1e-78e353911027 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:11:36.508968 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f4b1fa85-ddcb-4649-9a1e-78e353911027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:36.568434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1399/2807] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:11:36.578067 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a0fe9-b348-4578-8683-49ece6caaf64 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:36.579515 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-657a0fe9-b348-4578-8683-49ece6caaf64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:36.646446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a0fe9-b348-4578-8683-49ece6caaf64 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:36.650044 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c981080-2cad-473e-8a81-c4a9b895c036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:36.650687 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981080-2cad-473e-8a81-c4a9b895c036 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:36.650968 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981080-2cad-473e-8a81-c4a9b895c036 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:36.651244 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981080-2cad-473e-8a81-c4a9b895c036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:36.660491 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c981080-2cad-473e-8a81-c4a9b895c036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:36.661906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c981080-2cad-473e-8a81-c4a9b895c036 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:36.671241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1409/2808] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:11:36.949617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1fa85-ddcb-4649-9a1e-78e353911027 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:36.951787 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f4b1fa85-ddcb-4649-9a1e-78e353911027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:36.975950 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1fa85-ddcb-4649-9a1e-78e353911027 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:36.980699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:36.981587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:36.981884 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:36.982155 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] PATH_INFO: `/v3/projects/5bdc1c06b6894c928c3b2595cfa90ff8` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:36.983364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bdc1c06b6894c928c3b2595cfa90ff8)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:36.985342 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:37.030818 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:37.036776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bdc1c06b6894c928c3b2595cfa90ff8', 'request_id': 'req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.044526 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bdc1c06b6894c928c3b2595cfa90ff8', 'request_id': 'req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.050196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bdc1c06b6894c928c3b2595cfa90ff8', 'request_id': 'req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.061086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] The token cache is being invalidate because project 5bdc1c06b6894c928c3b2595cfa90ff8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:11:37.061372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.061996 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.064608 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1400/2809] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:11:36 2022] DELETE /identity/v3/projects/5bdc1c06b6894c928c3b2595cfa90ff8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:37.068742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c981080-2cad-473e-8a81-c4a9b895c036 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:37.078406 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c981080-2cad-473e-8a81-c4a9b895c036 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:37.117759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:37.139058 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0334036-71c7-48fe-9567-5736300898c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:37.139979 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0334036-71c7-48fe-9567-5736300898c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:37.140269 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0334036-71c7-48fe-9567-5736300898c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:37.140555 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0334036-71c7-48fe-9567-5736300898c8 admin admin] PATH_INFO: `/v3/users/70af958310044bd291fe46651e405f29` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:37.141752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0334036-71c7-48fe-9567-5736300898c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70af958310044bd291fe46651e405f29)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:37.143708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0334036-71c7-48fe-9567-5736300898c8 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:37.189427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e0334036-71c7-48fe-9567-5736300898c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70af958310044bd291fe46651e405f29'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.201196 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e0334036-71c7-48fe-9567-5736300898c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70af958310044bd291fe46651e405f29'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.208911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e0334036-71c7-48fe-9567-5736300898c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70af958310044bd291fe46651e405f29'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.209557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e0334036-71c7-48fe-9567-5736300898c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70af958310044bd291fe46651e405f29'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.216673 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e0334036-71c7-48fe-9567-5736300898c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70af958310044bd291fe46651e405f29'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.225076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1410/2810] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:11:37 2022] DELETE /identity/v3/users/70af958310044bd291fe46651e405f29 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:37.496779 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:37.498294 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:37.512194 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eede61d-2f0c-42a1-8c00-0f9eafaaa893 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:37.573090 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426dc24-970d-43e9-8cae-7583287edf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:37.573977 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426dc24-970d-43e9-8cae-7583287edf4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:37.574255 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426dc24-970d-43e9-8cae-7583287edf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:37.574531 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426dc24-970d-43e9-8cae-7583287edf4e admin admin] PATH_INFO: `/v3/projects/b20d336da9ff42d8972903dcc9d01e0a` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:37.575713 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426dc24-970d-43e9-8cae-7583287edf4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b20d336da9ff42d8972903dcc9d01e0a)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:37.577660 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426dc24-970d-43e9-8cae-7583287edf4e admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:37.628971 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f426dc24-970d-43e9-8cae-7583287edf4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b20d336da9ff42d8972903dcc9d01e0a', 'request_id': 'req-f426dc24-970d-43e9-8cae-7583287edf4e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.636270 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f426dc24-970d-43e9-8cae-7583287edf4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b20d336da9ff42d8972903dcc9d01e0a', 'request_id': 'req-f426dc24-970d-43e9-8cae-7583287edf4e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.641890 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f426dc24-970d-43e9-8cae-7583287edf4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b20d336da9ff42d8972903dcc9d01e0a', 'request_id': 'req-f426dc24-970d-43e9-8cae-7583287edf4e'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.652564 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f426dc24-970d-43e9-8cae-7583287edf4e admin admin] The token cache is being invalidate because project b20d336da9ff42d8972903dcc9d01e0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:11:37.652845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f426dc24-970d-43e9-8cae-7583287edf4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.653475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f426dc24-970d-43e9-8cae-7583287edf4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:37.654991 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1401/2811] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:11:37 2022] DELETE /identity/v3/projects/b20d336da9ff42d8972903dcc9d01e0a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:37.660628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c282197-b4b8-407f-80b5-2fb6d94c7090 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:37.661097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c282197-b4b8-407f-80b5-2fb6d94c7090 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:37.661200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c282197-b4b8-407f-80b5-2fb6d94c7090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:37.679133 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-3c282197-b4b8-407f-80b5-2fb6d94c7090 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:11:37.680392 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3c282197-b4b8-407f-80b5-2fb6d94c7090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:37.740806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1411/2812] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:11:37.748855 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dba391-34e7-4a9f-8696-ecefa6b21da2 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:37.749129 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dba391-34e7-4a9f-8696-ecefa6b21da2 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:37.749413 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dba391-34e7-4a9f-8696-ecefa6b21da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:37.768699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-16dba391-34e7-4a9f-8696-ecefa6b21da2 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:11:37.769908 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-16dba391-34e7-4a9f-8696-ecefa6b21da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:37.778381 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1402/2813] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:11:37.783101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c282197-b4b8-407f-80b5-2fb6d94c7090 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:37.793048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c282197-b4b8-407f-80b5-2fb6d94c7090 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:37.797612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02785d9-d430-4d9a-8854-c96563e2040f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:37.798270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02785d9-d430-4d9a-8854-c96563e2040f admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:37.798552 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02785d9-d430-4d9a-8854-c96563e2040f admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:37.798827 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02785d9-d430-4d9a-8854-c96563e2040f admin admin] PATH_INFO: `/v3/domains` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:37.799547 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02785d9-d430-4d9a-8854-c96563e2040f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91926) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:11:37.799856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02785d9-d430-4d9a-8854-c96563e2040f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:37.801470 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02785d9-d430-4d9a-8854-c96563e2040f admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:37.823414 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1412/2814] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:11:37.827573 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dba391-34e7-4a9f-8696-ecefa6b21da2 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:37.837395 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dba391-34e7-4a9f-8696-ecefa6b21da2 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:37.842096 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78de1c-cbe5-4198-9eae-1e44f2842c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:37.842745 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78de1c-cbe5-4198-9eae-1e44f2842c81 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:37.843027 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78de1c-cbe5-4198-9eae-1e44f2842c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:37.843303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d78de1c-cbe5-4198-9eae-1e44f2842c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:37.844024 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78de1c-cbe5-4198-9eae-1e44f2842c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:37.845939 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d78de1c-cbe5-4198-9eae-1e44f2842c81 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:37.872619 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1403/2815] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:11:37 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:11:37.878113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02785d9-d430-4d9a-8854-c96563e2040f admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:37.888179 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02785d9-d430-4d9a-8854-c96563e2040f admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:37.931527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:37.946229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c62615-0ddf-48af-ba1f-ab89caec5e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:37.946910 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c62615-0ddf-48af-ba1f-ab89caec5e1d admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:37.947199 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c62615-0ddf-48af-ba1f-ab89caec5e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:37.947486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c62615-0ddf-48af-ba1f-ab89caec5e1d admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:37.948232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c62615-0ddf-48af-ba1f-ab89caec5e1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:37.950181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c62615-0ddf-48af-ba1f-ab89caec5e1d admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:37.970272 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1413/2816] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:11:37 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:11:37.974507 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78de1c-cbe5-4198-9eae-1e44f2842c81 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:37.984806 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d78de1c-cbe5-4198-9eae-1e44f2842c81 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:37.989918 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b24677-5165-4cec-9884-06ba7d0e36e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:37.990580 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b24677-5165-4cec-9884-06ba7d0e36e2 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:37.990868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b24677-5165-4cec-9884-06ba7d0e36e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:37.991149 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b24677-5165-4cec-9884-06ba7d0e36e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:37.991880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b24677-5165-4cec-9884-06ba7d0e36e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:37.993500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b24677-5165-4cec-9884-06ba7d0e36e2 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:38.004562 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1404/2817] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:11:38.008416 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c62615-0ddf-48af-ba1f-ab89caec5e1d admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:38.018734 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c62615-0ddf-48af-ba1f-ab89caec5e1d admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:38.023410 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cec522e-812d-4399-bcd8-b00596e63dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:38.024107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec522e-812d-4399-bcd8-b00596e63dcf admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:38.024401 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec522e-812d-4399-bcd8-b00596e63dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:38.024685 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cec522e-812d-4399-bcd8-b00596e63dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:38.025414 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec522e-812d-4399-bcd8-b00596e63dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:38.027006 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cec522e-812d-4399-bcd8-b00596e63dcf admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:38.037357 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1414/2818] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:11:38.041774 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b24677-5165-4cec-9884-06ba7d0e36e2 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:38.052205 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b24677-5165-4cec-9884-06ba7d0e36e2 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:38.056743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0905608-a1ce-41dc-b76a-55a3f054a536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:38.057613 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0905608-a1ce-41dc-b76a-55a3f054a536 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:38.057893 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0905608-a1ce-41dc-b76a-55a3f054a536 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:38.058169 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0905608-a1ce-41dc-b76a-55a3f054a536 admin admin] PATH_INFO: `/v3/projects/e9ca2ae90bc941a3ab1fe2600830c894/users/ec4be02f7cd94382a2960d270cb5b99d/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:38.073841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0905608-a1ce-41dc-b76a-55a3f054a536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9ca2ae90bc941a3ab1fe2600830c894, user_id=ec4be02f7cd94382a2960d270cb5b99d, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:38.076550 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0905608-a1ce-41dc-b76a-55a3f054a536 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:38.084770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:38.091131 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1405/2819] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:11:38 2022] PUT /identity/v3/projects/e9ca2ae90bc941a3ab1fe2600830c894/users/ec4be02f7cd94382a2960d270cb5b99d/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:38.101858 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370c1b9-2f8b-4d08-89fc-f1ee7e870884 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:38.102153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370c1b9-2f8b-4d08-89fc-f1ee7e870884 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:38.102437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f370c1b9-2f8b-4d08-89fc-f1ee7e870884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:38.129313 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-f370c1b9-2f8b-4d08-89fc-f1ee7e870884 None None] MFA Rules not processed for user `ec4be02f7cd94382a2960d270cb5b99d`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:11:38.130524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-f370c1b9-2f8b-4d08-89fc-f1ee7e870884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:38.194605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1415/2820] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:11:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:11:38.198998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0905608-a1ce-41dc-b76a-55a3f054a536 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:38.269600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0905608-a1ce-41dc-b76a-55a3f054a536 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:38.274200 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8247380-c99c-48f5-bfbd-2efa8bec9793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:38.274855 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8247380-c99c-48f5-bfbd-2efa8bec9793 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:38.275137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8247380-c99c-48f5-bfbd-2efa8bec9793 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:38.275413 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8247380-c99c-48f5-bfbd-2efa8bec9793 admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:38.276135 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8247380-c99c-48f5-bfbd-2efa8bec9793 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:38.278029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8247380-c99c-48f5-bfbd-2efa8bec9793 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:38.303771 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1406/2821] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:11:38 2022] POST /identity/v3/projects => generated 377 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:11:38.307862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370c1b9-2f8b-4d08-89fc-f1ee7e870884 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:38.319110 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f370c1b9-2f8b-4d08-89fc-f1ee7e870884 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:38.374604 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda49ea9-177e-4dcb-ad07-b2b602861d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:38.375276 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda49ea9-177e-4dcb-ad07-b2b602861d4e admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:38.376423 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda49ea9-177e-4dcb-ad07-b2b602861d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:38.376887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda49ea9-177e-4dcb-ad07-b2b602861d4e admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:38.377603 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda49ea9-177e-4dcb-ad07-b2b602861d4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:38.379551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda49ea9-177e-4dcb-ad07-b2b602861d4e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:38.405537 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1416/2822] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:11:38 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:11:38.409689 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8247380-c99c-48f5-bfbd-2efa8bec9793 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:38.420095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8247380-c99c-48f5-bfbd-2efa8bec9793 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:38.424755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add9f64-e72f-4e36-9216-e985b46f6796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:38.425416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add9f64-e72f-4e36-9216-e985b46f6796 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:38.425701 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add9f64-e72f-4e36-9216-e985b46f6796 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:38.425976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4add9f64-e72f-4e36-9216-e985b46f6796 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:38.426707 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add9f64-e72f-4e36-9216-e985b46f6796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:38.428330 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4add9f64-e72f-4e36-9216-e985b46f6796 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:38.439279 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1407/2823] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:11:38.443180 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda49ea9-177e-4dcb-ad07-b2b602861d4e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:38.453146 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda49ea9-177e-4dcb-ad07-b2b602861d4e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:38.458535 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366ce85-65f7-4220-b711-7b07c41eda14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:38.459275 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366ce85-65f7-4220-b711-7b07c41eda14 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:38.459509 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366ce85-65f7-4220-b711-7b07c41eda14 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:38.459756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366ce85-65f7-4220-b711-7b07c41eda14 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:38.460507 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366ce85-65f7-4220-b711-7b07c41eda14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:38.462102 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366ce85-65f7-4220-b711-7b07c41eda14 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:38.472118 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1417/2824] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:11:38.476345 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add9f64-e72f-4e36-9216-e985b46f6796 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:38.486134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4add9f64-e72f-4e36-9216-e985b46f6796 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:38.490600 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a23d3-271a-4983-9ca2-c885d762ff37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:38.491471 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a23d3-271a-4983-9ca2-c885d762ff37 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:38.491749 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a23d3-271a-4983-9ca2-c885d762ff37 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:38.492056 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a23d3-271a-4983-9ca2-c885d762ff37 admin admin] PATH_INFO: `/v3/projects/863012263b9e4a5a9e247cdf90dac44f/users/52275b74b95d4160baab4ba448992837/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:38.506750 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a23d3-271a-4983-9ca2-c885d762ff37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=863012263b9e4a5a9e247cdf90dac44f, user_id=52275b74b95d4160baab4ba448992837, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:38.509404 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a23d3-271a-4983-9ca2-c885d762ff37 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:38.523431 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1408/2825] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:11:38 2022] PUT /identity/v3/projects/863012263b9e4a5a9e247cdf90dac44f/users/52275b74b95d4160baab4ba448992837/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:38.535885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278040a-0d5f-4a8b-a89a-b7b8de2a3570 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:38.536229 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278040a-0d5f-4a8b-a89a-b7b8de2a3570 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:38.536538 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278040a-0d5f-4a8b-a89a-b7b8de2a3570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:38.563209 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b278040a-0d5f-4a8b-a89a-b7b8de2a3570 None None] MFA Rules not processed for user `52275b74b95d4160baab4ba448992837`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:11:38.564462 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b278040a-0d5f-4a8b-a89a-b7b8de2a3570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:38.583831 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:38.628988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1418/2826] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:11:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:11:38.633593 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a23d3-271a-4983-9ca2-c885d762ff37 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:38.695066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a23d3-271a-4983-9ca2-c885d762ff37 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:38.699770 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dd85c-1a8c-4ecb-9adc-adc51421676e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:38.700458 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dd85c-1a8c-4ecb-9adc-adc51421676e admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:38.700746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dd85c-1a8c-4ecb-9adc-adc51421676e admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:38.701025 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dd85c-1a8c-4ecb-9adc-adc51421676e admin admin] PATH_INFO: `/v3/projects` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:38.701730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67dd85c-1a8c-4ecb-9adc-adc51421676e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:38.704235 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67dd85c-1a8c-4ecb-9adc-adc51421676e admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:38.729711 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1409/2827] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:11:38 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:11:38.733911 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278040a-0d5f-4a8b-a89a-b7b8de2a3570 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:38.745670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278040a-0d5f-4a8b-a89a-b7b8de2a3570 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:38.801477 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c988a-0a7a-4fdf-8de1-bac4a7f7cbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:38.802153 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c988a-0a7a-4fdf-8de1-bac4a7f7cbfd admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:38.802912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c988a-0a7a-4fdf-8de1-bac4a7f7cbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:38.802912 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461c988a-0a7a-4fdf-8de1-bac4a7f7cbfd admin admin] PATH_INFO: `/v3/users` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:38.803404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c988a-0a7a-4fdf-8de1-bac4a7f7cbfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:38.805350 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461c988a-0a7a-4fdf-8de1-bac4a7f7cbfd admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:38.825614 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1419/2828] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:11:38 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:11:38.829744 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dd85c-1a8c-4ecb-9adc-adc51421676e admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:38.840330 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67dd85c-1a8c-4ecb-9adc-adc51421676e admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:38.845031 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db28364-cfba-4b80-9228-30f7f0b95051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:38.845697 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db28364-cfba-4b80-9228-30f7f0b95051 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:38.845982 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db28364-cfba-4b80-9228-30f7f0b95051 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:38.846259 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db28364-cfba-4b80-9228-30f7f0b95051 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:38.846986 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db28364-cfba-4b80-9228-30f7f0b95051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:38.848625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db28364-cfba-4b80-9228-30f7f0b95051 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:38.859273 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1410/2829] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:11:38.864886 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c988a-0a7a-4fdf-8de1-bac4a7f7cbfd admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:38.875060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461c988a-0a7a-4fdf-8de1-bac4a7f7cbfd admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:38.880289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3464c6-7f1d-4d39-9c4c-d56dc207ccf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:38.881169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3464c6-7f1d-4d39-9c4c-d56dc207ccf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:38.881458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3464c6-7f1d-4d39-9c4c-d56dc207ccf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:38.881734 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3464c6-7f1d-4d39-9c4c-d56dc207ccf3 admin admin] PATH_INFO: `/v3/projects/e7ef3977168642b6864ed26410b28784/users/2a3e66a8aed948ceb543b447788ecc50/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:38.896027 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3464c6-7f1d-4d39-9c4c-d56dc207ccf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ef3977168642b6864ed26410b28784, user_id=2a3e66a8aed948ceb543b447788ecc50, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:38.898649 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3464c6-7f1d-4d39-9c4c-d56dc207ccf3 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:38.912628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1420/2830] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:11:38 2022] PUT /identity/v3/projects/e7ef3977168642b6864ed26410b28784/users/2a3e66a8aed948ceb543b447788ecc50/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:38.916505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db28364-cfba-4b80-9228-30f7f0b95051 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:38.926845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db28364-cfba-4b80-9228-30f7f0b95051 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:38.945386 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:38.986309 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4eb9c-1e5d-481d-b6a9-f817ead21da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:38.986902 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4eb9c-1e5d-481d-b6a9-f817ead21da4 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:38.987191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4eb9c-1e5d-481d-b6a9-f817ead21da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:38.987475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4eb9c-1e5d-481d-b6a9-f817ead21da4 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:38.988236 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4eb9c-1e5d-481d-b6a9-f817ead21da4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:38.989828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4eb9c-1e5d-481d-b6a9-f817ead21da4 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:38.999481 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1411/2831] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:11:39.004815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3464c6-7f1d-4d39-9c4c-d56dc207ccf3 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:39.014842 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3464c6-7f1d-4d39-9c4c-d56dc207ccf3 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:39.020187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf35e98-b230-4330-b9f3-fc94922a2103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:39.021076 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf35e98-b230-4330-b9f3-fc94922a2103 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:39.021359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf35e98-b230-4330-b9f3-fc94922a2103 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:39.021639 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf35e98-b230-4330-b9f3-fc94922a2103 admin admin] PATH_INFO: `/v3/projects/e7ef3977168642b6864ed26410b28784/users/2a3e66a8aed948ceb543b447788ecc50/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:39.023614 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf35e98-b230-4330-b9f3-fc94922a2103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7ef3977168642b6864ed26410b28784, user_id=2a3e66a8aed948ceb543b447788ecc50, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:39.026373 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf35e98-b230-4330-b9f3-fc94922a2103 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:39.036966 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1421/2832] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:11:39 2022] PUT /identity/v3/projects/e7ef3977168642b6864ed26410b28784/users/2a3e66a8aed948ceb543b447788ecc50/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:39.047020 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135d909-3d28-49fc-9779-5ced827b3dd8 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:39.047344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135d909-3d28-49fc-9779-5ced827b3dd8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:39.047631 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135d909-3d28-49fc-9779-5ced827b3dd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:39.074308 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-3135d909-3d28-49fc-9779-5ced827b3dd8 None None] MFA Rules not processed for user `2a3e66a8aed948ceb543b447788ecc50`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:11:39.075525 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3135d909-3d28-49fc-9779-5ced827b3dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:39.144660 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1412/2833] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:11:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:11:39.155777 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf35e98-b230-4330-b9f3-fc94922a2103 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:39.157803 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-baf35e98-b230-4330-b9f3-fc94922a2103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:39.214259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf35e98-b230-4330-b9f3-fc94922a2103 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:39.217974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f1283-a13c-47dc-b3e5-a9a08b4c1bfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:39.218623 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f1283-a13c-47dc-b3e5-a9a08b4c1bfa service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:39.218913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f1283-a13c-47dc-b3e5-a9a08b4c1bfa service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:39.219215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f1283-a13c-47dc-b3e5-a9a08b4c1bfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:39.248664 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:39.277986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f1283-a13c-47dc-b3e5-a9a08b4c1bfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:39.279422 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f1283-a13c-47dc-b3e5-a9a08b4c1bfa service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:39.288756 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1422/2834] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:11:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:11:40.760860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135d909-3d28-49fc-9779-5ced827b3dd8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:40.762249 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-3135d909-3d28-49fc-9779-5ced827b3dd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:40.774598 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135d909-3d28-49fc-9779-5ced827b3dd8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:40.778190 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27366823-564d-4539-a5a9-a8d69c6dd95a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:40.778837 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27366823-564d-4539-a5a9-a8d69c6dd95a service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:40.779124 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27366823-564d-4539-a5a9-a8d69c6dd95a service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:40.779403 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27366823-564d-4539-a5a9-a8d69c6dd95a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:40.836713 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27366823-564d-4539-a5a9-a8d69c6dd95a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:40.838079 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27366823-564d-4539-a5a9-a8d69c6dd95a service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:40.847392 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1413/2835] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:11:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:11:41.355120 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f1283-a13c-47dc-b3e5-a9a08b4c1bfa service cinder] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:41.363787 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f1283-a13c-47dc-b3e5-a9a08b4c1bfa service cinder] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:41.367425 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08bce3d-9c4b-482f-95d1-926abecfc303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:41.368114 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bce3d-9c4b-482f-95d1-926abecfc303 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:41.368404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bce3d-9c4b-482f-95d1-926abecfc303 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:41.368685 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bce3d-9c4b-482f-95d1-926abecfc303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:41.378113 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08bce3d-9c4b-482f-95d1-926abecfc303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:41.379527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08bce3d-9c4b-482f-95d1-926abecfc303 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:41.388903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1423/2836] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:11:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:11:45.227012 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27366823-564d-4539-a5a9-a8d69c6dd95a service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:45.270668 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=100 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:45.288831 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27366823-564d-4539-a5a9-a8d69c6dd95a service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:45.293264 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5738848-ba4b-468f-a3ee-f1a12211e5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:45.294126 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5738848-ba4b-468f-a3ee-f1a12211e5ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:45.294401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5738848-ba4b-468f-a3ee-f1a12211e5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:45.294669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5738848-ba4b-468f-a3ee-f1a12211e5ed admin admin] PATH_INFO: `/v3/users/ec4be02f7cd94382a2960d270cb5b99d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:45.295845 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5738848-ba4b-468f-a3ee-f1a12211e5ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4be02f7cd94382a2960d270cb5b99d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:45.297781 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5738848-ba4b-468f-a3ee-f1a12211e5ed admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:45.343841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d5738848-ba4b-468f-a3ee-f1a12211e5ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4be02f7cd94382a2960d270cb5b99d'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:45.356172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d5738848-ba4b-468f-a3ee-f1a12211e5ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4be02f7cd94382a2960d270cb5b99d'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:45.356888 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d5738848-ba4b-468f-a3ee-f1a12211e5ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4be02f7cd94382a2960d270cb5b99d'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:45.364588 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d5738848-ba4b-468f-a3ee-f1a12211e5ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4be02f7cd94382a2960d270cb5b99d'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:45.365208 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-d5738848-ba4b-468f-a3ee-f1a12211e5ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4be02f7cd94382a2960d270cb5b99d'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:45.378062 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1414/2837] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:11:45 2022] DELETE /identity/v3/users/ec4be02f7cd94382a2960d270cb5b99d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:45.382524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af15dac-31e0-4ea2-a70a-73de61ed6e4c None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:45.382995 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af15dac-31e0-4ea2-a70a-73de61ed6e4c None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:45.383101 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af15dac-31e0-4ea2-a70a-73de61ed6e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:45.399864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-0af15dac-31e0-4ea2-a70a-73de61ed6e4c None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:11:45.401151 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0af15dac-31e0-4ea2-a70a-73de61ed6e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:45.459948 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1424/2838] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:11:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:11:45.469592 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5738848-ba4b-468f-a3ee-f1a12211e5ed admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:45.471004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-d5738848-ba4b-468f-a3ee-f1a12211e5ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:45.527055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5738848-ba4b-468f-a3ee-f1a12211e5ed admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:45.531026 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924b2ad-3b2b-4a78-82e6-58b27b42e22e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:45.531869 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924b2ad-3b2b-4a78-82e6-58b27b42e22e service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:45.532101 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924b2ad-3b2b-4a78-82e6-58b27b42e22e service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:45.532377 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924b2ad-3b2b-4a78-82e6-58b27b42e22e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:45.541925 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924b2ad-3b2b-4a78-82e6-58b27b42e22e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:45.543357 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924b2ad-3b2b-4a78-82e6-58b27b42e22e service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:45.554501 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1415/2839] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:11:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:11:45.827622 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af15dac-31e0-4ea2-a70a-73de61ed6e4c None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:45.829601 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-0af15dac-31e0-4ea2-a70a-73de61ed6e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:45.841657 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af15dac-31e0-4ea2-a70a-73de61ed6e4c None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:45.846286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:45.847287 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:45.847703 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:45.847915 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] PATH_INFO: `/v3/projects/e9ca2ae90bc941a3ab1fe2600830c894` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:45.849199 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9ca2ae90bc941a3ab1fe2600830c894)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:45.851213 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:45.898807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9ca2ae90bc941a3ab1fe2600830c894', 'request_id': 'req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:45.907109 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9ca2ae90bc941a3ab1fe2600830c894', 'request_id': 'req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:45.913322 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9ca2ae90bc941a3ab1fe2600830c894', 'request_id': 'req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:45.923505 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] The token cache is being invalidate because project e9ca2ae90bc941a3ab1fe2600830c894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:11:45.923795 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:45.924471 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:45.926041 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1425/2840] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:11:45 2022] DELETE /identity/v3/projects/e9ca2ae90bc941a3ab1fe2600830c894 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:45.930038 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924b2ad-3b2b-4a78-82e6-58b27b42e22e service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:45.940214 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924b2ad-3b2b-4a78-82e6-58b27b42e22e service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:45.997338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6734e-db7e-4982-82ef-122691561f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:45.998212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6734e-db7e-4982-82ef-122691561f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:45.998501 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6734e-db7e-4982-82ef-122691561f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:45.998778 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd6734e-db7e-4982-82ef-122691561f09 admin admin] PATH_INFO: `/v3/users/52275b74b95d4160baab4ba448992837` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:45.999995 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6734e-db7e-4982-82ef-122691561f09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52275b74b95d4160baab4ba448992837)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:46.002509 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd6734e-db7e-4982-82ef-122691561f09 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:46.007007 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:46.050769 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-5dd6734e-db7e-4982-82ef-122691561f09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52275b74b95d4160baab4ba448992837'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.062626 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-5dd6734e-db7e-4982-82ef-122691561f09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52275b74b95d4160baab4ba448992837'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.063863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-5dd6734e-db7e-4982-82ef-122691561f09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52275b74b95d4160baab4ba448992837'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.071281 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-5dd6734e-db7e-4982-82ef-122691561f09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52275b74b95d4160baab4ba448992837'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.071901 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-5dd6734e-db7e-4982-82ef-122691561f09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52275b74b95d4160baab4ba448992837'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.084659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1416/2841] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:11:45 2022] DELETE /identity/v3/users/52275b74b95d4160baab4ba448992837 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:46.357241 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:46.358680 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:46.374422 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ddb39-46fe-4bbd-a0c9-ffc696473e3e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:46.413744 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:46.434056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:46.434951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:46.435230 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:46.435503 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] PATH_INFO: `/v3/projects/863012263b9e4a5a9e247cdf90dac44f` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:46.436779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] RBAC: Authorizing `identity:delete_project(project_id=863012263b9e4a5a9e247cdf90dac44f)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:46.439162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:46.484107 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '863012263b9e4a5a9e247cdf90dac44f', 'request_id': 'req-d91e3f29-836e-4417-9a3b-0a6c89ec369c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.491854 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '863012263b9e4a5a9e247cdf90dac44f', 'request_id': 'req-d91e3f29-836e-4417-9a3b-0a6c89ec369c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.498033 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '863012263b9e4a5a9e247cdf90dac44f', 'request_id': 'req-d91e3f29-836e-4417-9a3b-0a6c89ec369c'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.508421 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] The token cache is being invalidate because project 863012263b9e4a5a9e247cdf90dac44f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:11:46.508711 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.509362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.510906 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1426/2842] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:11:46 2022] DELETE /identity/v3/projects/863012263b9e4a5a9e247cdf90dac44f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:46.514853 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6734e-db7e-4982-82ef-122691561f09 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:46.516270 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5dd6734e-db7e-4982-82ef-122691561f09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:46.528813 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd6734e-db7e-4982-82ef-122691561f09 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:46.585011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a1d659-ae96-4cf2-ab7f-cb92a3f425b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:46.585887 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1d659-ae96-4cf2-ab7f-cb92a3f425b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:46.586173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1d659-ae96-4cf2-ab7f-cb92a3f425b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:46.586456 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a1d659-ae96-4cf2-ab7f-cb92a3f425b7 admin admin] PATH_INFO: `/v3/users/2a3e66a8aed948ceb543b447788ecc50` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:46.587645 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1d659-ae96-4cf2-ab7f-cb92a3f425b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a3e66a8aed948ceb543b447788ecc50)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:46.589587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a1d659-ae96-4cf2-ab7f-cb92a3f425b7 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:46.635156 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e4a1d659-ae96-4cf2-ab7f-cb92a3f425b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a3e66a8aed948ceb543b447788ecc50'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.646926 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e4a1d659-ae96-4cf2-ab7f-cb92a3f425b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a3e66a8aed948ceb543b447788ecc50'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.647607 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e4a1d659-ae96-4cf2-ab7f-cb92a3f425b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a3e66a8aed948ceb543b447788ecc50'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.654277 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e4a1d659-ae96-4cf2-ab7f-cb92a3f425b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a3e66a8aed948ceb543b447788ecc50'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.655094 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-e4a1d659-ae96-4cf2-ab7f-cb92a3f425b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a3e66a8aed948ceb543b447788ecc50'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:46.668942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1417/2843] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:11:46 2022] DELETE /identity/v3/users/2a3e66a8aed948ceb543b447788ecc50 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:46.945524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:46.947020 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:46.960566 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91e3f29-836e-4417-9a3b-0a6c89ec369c admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:47.018670 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fd285-e7d0-4042-b815-684922fd3bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:47.019464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fd285-e7d0-4042-b815-684922fd3bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:47.019749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fd285-e7d0-4042-b815-684922fd3bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:47.020052 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fd285-e7d0-4042-b815-684922fd3bf8 admin admin] PATH_INFO: `/v3/projects/e7ef3977168642b6864ed26410b28784` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:47.021220 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fd285-e7d0-4042-b815-684922fd3bf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7ef3977168642b6864ed26410b28784)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:47.023135 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fd285-e7d0-4042-b815-684922fd3bf8 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:47.069283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5e1fd285-e7d0-4042-b815-684922fd3bf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7ef3977168642b6864ed26410b28784', 'request_id': 'req-5e1fd285-e7d0-4042-b815-684922fd3bf8'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:47.075802 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5e1fd285-e7d0-4042-b815-684922fd3bf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7ef3977168642b6864ed26410b28784', 'request_id': 'req-5e1fd285-e7d0-4042-b815-684922fd3bf8'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:47.082060 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5e1fd285-e7d0-4042-b815-684922fd3bf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7ef3977168642b6864ed26410b28784', 'request_id': 'req-5e1fd285-e7d0-4042-b815-684922fd3bf8'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:47.089030 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:47.093453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5e1fd285-e7d0-4042-b815-684922fd3bf8 admin admin] The token cache is being invalidate because project e7ef3977168642b6864ed26410b28784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:11:47.093741 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5e1fd285-e7d0-4042-b815-684922fd3bf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:47.094409 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-5e1fd285-e7d0-4042-b815-684922fd3bf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:11:47.096418 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1427/2844] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:11:46 2022] DELETE /identity/v3/projects/e7ef3977168642b6864ed26410b28784 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:47.101442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae3c82-654f-4f22-a0f2-7d3c667f0abe None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:47.101734 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae3c82-654f-4f22-a0f2-7d3c667f0abe None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:47.102016 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fae3c82-654f-4f22-a0f2-7d3c667f0abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:47.108338 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:47.123834 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-9fae3c82-654f-4f22-a0f2-7d3c667f0abe None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:11:47.125074 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-9fae3c82-654f-4f22-a0f2-7d3c667f0abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:47.184210 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1418/2845] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:11:47.193694 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9431b-9f45-4a46-b611-80b561c2e794 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:47.193988 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9431b-9f45-4a46-b611-80b561c2e794 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:47.194273 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9431b-9f45-4a46-b611-80b561c2e794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:47.212126 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-68e9431b-9f45-4a46-b611-80b561c2e794 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:11:47.213333 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-68e9431b-9f45-4a46-b611-80b561c2e794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:47.221681 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1428/2846] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:11:47.225861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae3c82-654f-4f22-a0f2-7d3c667f0abe None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:47.236364 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fae3c82-654f-4f22-a0f2-7d3c667f0abe None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:47.240920 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30479204-06f6-4f31-bab8-81f02a940f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:47.241572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479204-06f6-4f31-bab8-81f02a940f62 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:47.241862 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479204-06f6-4f31-bab8-81f02a940f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:47.242146 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30479204-06f6-4f31-bab8-81f02a940f62 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:47.243076 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30479204-06f6-4f31-bab8-81f02a940f62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:11:47.243389 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30479204-06f6-4f31-bab8-81f02a940f62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:47.245015 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30479204-06f6-4f31-bab8-81f02a940f62 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:47.256609 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1419/2847] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:11:47.260752 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9431b-9f45-4a46-b611-80b561c2e794 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:47.269950 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9431b-9f45-4a46-b611-80b561c2e794 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:47.274613 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf473e-eeb1-41da-9e1c-cfcf7cf98147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:47.275262 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf473e-eeb1-41da-9e1c-cfcf7cf98147 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:47.275551 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf473e-eeb1-41da-9e1c-cfcf7cf98147 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:47.275830 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf473e-eeb1-41da-9e1c-cfcf7cf98147 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:47.276559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf473e-eeb1-41da-9e1c-cfcf7cf98147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:47.278458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaf473e-eeb1-41da-9e1c-cfcf7cf98147 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:47.304986 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1429/2848] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:11:47 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:11:47.308883 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30479204-06f6-4f31-bab8-81f02a940f62 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:47.318951 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30479204-06f6-4f31-bab8-81f02a940f62 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:47.374688 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6bff0-ca5b-41d0-bd33-8715baa5298d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:47.375358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6bff0-ca5b-41d0-bd33-8715baa5298d admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:47.375672 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6bff0-ca5b-41d0-bd33-8715baa5298d admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:47.375970 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e6bff0-ca5b-41d0-bd33-8715baa5298d admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:47.376660 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6bff0-ca5b-41d0-bd33-8715baa5298d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:47.378566 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e6bff0-ca5b-41d0-bd33-8715baa5298d admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:47.400063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1420/2849] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:11:47 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:11:47.404458 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf473e-eeb1-41da-9e1c-cfcf7cf98147 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:47.415646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf473e-eeb1-41da-9e1c-cfcf7cf98147 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:47.420407 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4a85e-b91d-4fd4-8ffd-f67a251840cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:47.421068 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4a85e-b91d-4fd4-8ffd-f67a251840cc admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:47.421357 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4a85e-b91d-4fd4-8ffd-f67a251840cc admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:47.421640 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de4a85e-b91d-4fd4-8ffd-f67a251840cc admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:47.422391 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4a85e-b91d-4fd4-8ffd-f67a251840cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:47.424560 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de4a85e-b91d-4fd4-8ffd-f67a251840cc admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:47.435200 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1430/2850] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:11:47.439162 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6bff0-ca5b-41d0-bd33-8715baa5298d admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:47.449011 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e6bff0-ca5b-41d0-bd33-8715baa5298d admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:47.454290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbb3c1-5738-4363-bfbe-a3f9d26aa9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:47.454990 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbb3c1-5738-4363-bfbe-a3f9d26aa9bf admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:47.455277 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbb3c1-5738-4363-bfbe-a3f9d26aa9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:47.455559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbb3c1-5738-4363-bfbe-a3f9d26aa9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:47.456321 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbb3c1-5738-4363-bfbe-a3f9d26aa9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:47.457970 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbb3c1-5738-4363-bfbe-a3f9d26aa9bf admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:47.468678 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1421/2851] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:11:47.473264 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4a85e-b91d-4fd4-8ffd-f67a251840cc admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:47.483426 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de4a85e-b91d-4fd4-8ffd-f67a251840cc admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:47.488587 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab36ee-9092-47f9-b557-3e348fca0dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:47.489454 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab36ee-9092-47f9-b557-3e348fca0dea admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:47.489739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab36ee-9092-47f9-b557-3e348fca0dea admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:47.490015 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab36ee-9092-47f9-b557-3e348fca0dea admin admin] PATH_INFO: `/v3/projects/6ea34277ced644d19c3746a67d5b8788/users/05b0ed3671274e7a97bff754d000b29f/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:47.504382 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab36ee-9092-47f9-b557-3e348fca0dea admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea34277ced644d19c3746a67d5b8788, user_id=05b0ed3671274e7a97bff754d000b29f, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:47.507024 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab36ee-9092-47f9-b557-3e348fca0dea admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:47.520253 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1431/2852] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:11:47 2022] PUT /identity/v3/projects/6ea34277ced644d19c3746a67d5b8788/users/05b0ed3671274e7a97bff754d000b29f/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:11:47.581497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139026db-6df8-4948-bb3d-4dc8da82deb5 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:47.582053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139026db-6df8-4948-bb3d-4dc8da82deb5 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:47.582053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139026db-6df8-4948-bb3d-4dc8da82deb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:47.610666 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-139026db-6df8-4948-bb3d-4dc8da82deb5 None None] MFA Rules not processed for user `05b0ed3671274e7a97bff754d000b29f`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:11:47.611911 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-139026db-6df8-4948-bb3d-4dc8da82deb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:47.671643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:47.677372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1422/2853] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:11:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:11:47.688525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab36ee-9092-47f9-b557-3e348fca0dea admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:11:47.690224 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-08ab36ee-9092-47f9-b557-3e348fca0dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:11:47.747901 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab36ee-9092-47f9-b557-3e348fca0dea admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:11:47.751525 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82acf52b-9df6-41f8-a45b-0ce58eda5637 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:11:47.752214 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acf52b-9df6-41f8-a45b-0ce58eda5637 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:11:47.752507 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acf52b-9df6-41f8-a45b-0ce58eda5637 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:11:47.752786 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82acf52b-9df6-41f8-a45b-0ce58eda5637 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:11:47.761080 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82acf52b-9df6-41f8-a45b-0ce58eda5637 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:11:47.762507 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82acf52b-9df6-41f8-a45b-0ce58eda5637 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:11:47.772033 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1432/2854] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:11:47 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:11:57.678355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:11:57.767931 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=53,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:12:29.897444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139026db-6df8-4948-bb3d-4dc8da82deb5 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:29.959542 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139026db-6df8-4948-bb3d-4dc8da82deb5 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:29.964019 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a1f6c-0824-4de4-8eae-e9e44c8dc135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:29.964903 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a1f6c-0824-4de4-8eae-e9e44c8dc135 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:29.965201 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a1f6c-0824-4de4-8eae-e9e44c8dc135 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:29.965474 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a1f6c-0824-4de4-8eae-e9e44c8dc135 admin admin] PATH_INFO: `/v3/users/05b0ed3671274e7a97bff754d000b29f` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:29.967024 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a1f6c-0824-4de4-8eae-e9e44c8dc135 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b0ed3671274e7a97bff754d000b29f)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:29.969055 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a1f6c-0824-4de4-8eae-e9e44c8dc135 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:30.014716 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f75a1f6c-0824-4de4-8eae-e9e44c8dc135 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b0ed3671274e7a97bff754d000b29f'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:30.027284 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f75a1f6c-0824-4de4-8eae-e9e44c8dc135 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b0ed3671274e7a97bff754d000b29f'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:30.028021 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f75a1f6c-0824-4de4-8eae-e9e44c8dc135 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b0ed3671274e7a97bff754d000b29f'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:30.036127 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f75a1f6c-0824-4de4-8eae-e9e44c8dc135 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b0ed3671274e7a97bff754d000b29f'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:30.036741 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-f75a1f6c-0824-4de4-8eae-e9e44c8dc135 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b0ed3671274e7a97bff754d000b29f'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:30.048830 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1423/2855] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:12:29 2022] DELETE /identity/v3/users/05b0ed3671274e7a97bff754d000b29f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:12:30.053376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a767a-c364-4f68-8555-6db2d41f05a7 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:30.054042 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a767a-c364-4f68-8555-6db2d41f05a7 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:30.054042 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a767a-c364-4f68-8555-6db2d41f05a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:30.070473 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b70a767a-c364-4f68-8555-6db2d41f05a7 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:12:30.071664 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b70a767a-c364-4f68-8555-6db2d41f05a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:30.130933 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1433/2856] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:12:30.140467 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a1f6c-0824-4de4-8eae-e9e44c8dc135 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:30.141892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-f75a1f6c-0824-4de4-8eae-e9e44c8dc135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:30.199147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a1f6c-0824-4de4-8eae-e9e44c8dc135 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:30.202795 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5160f49-d8e1-4db2-a8a5-aaf1cf4f7c1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:30.203444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5160f49-d8e1-4db2-a8a5-aaf1cf4f7c1a service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:30.203729 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5160f49-d8e1-4db2-a8a5-aaf1cf4f7c1a service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:30.204043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5160f49-d8e1-4db2-a8a5-aaf1cf4f7c1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:30.213438 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5160f49-d8e1-4db2-a8a5-aaf1cf4f7c1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:30.214860 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5160f49-d8e1-4db2-a8a5-aaf1cf4f7c1a service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:30.224932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1424/2857] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:12:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:12:30.487860 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a767a-c364-4f68-8555-6db2d41f05a7 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:30.489444 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b70a767a-c364-4f68-8555-6db2d41f05a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:30.501977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a767a-c364-4f68-8555-6db2d41f05a7 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:30.507195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2850b2-5265-4c02-97cf-20f2f7a37809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:30.508130 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2850b2-5265-4c02-97cf-20f2f7a37809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:30.512453 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2850b2-5265-4c02-97cf-20f2f7a37809 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:30.512762 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2850b2-5265-4c02-97cf-20f2f7a37809 admin admin] PATH_INFO: `/v3/projects/6ea34277ced644d19c3746a67d5b8788` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:30.514088 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2850b2-5265-4c02-97cf-20f2f7a37809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ea34277ced644d19c3746a67d5b8788)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:30.516849 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2850b2-5265-4c02-97cf-20f2f7a37809 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:30.566463 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3a2850b2-5265-4c02-97cf-20f2f7a37809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ea34277ced644d19c3746a67d5b8788', 'request_id': 'req-3a2850b2-5265-4c02-97cf-20f2f7a37809'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:30.573524 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3a2850b2-5265-4c02-97cf-20f2f7a37809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ea34277ced644d19c3746a67d5b8788', 'request_id': 'req-3a2850b2-5265-4c02-97cf-20f2f7a37809'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:30.580342 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3a2850b2-5265-4c02-97cf-20f2f7a37809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ea34277ced644d19c3746a67d5b8788', 'request_id': 'req-3a2850b2-5265-4c02-97cf-20f2f7a37809'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:30.590366 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3a2850b2-5265-4c02-97cf-20f2f7a37809 admin admin] The token cache is being invalidate because project 6ea34277ced644d19c3746a67d5b8788 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:12:30.590648 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3a2850b2-5265-4c02-97cf-20f2f7a37809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:30.591280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-3a2850b2-5265-4c02-97cf-20f2f7a37809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:30.592856 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1434/2858] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:12:30 2022] DELETE /identity/v3/projects/6ea34277ced644d19c3746a67d5b8788 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:12:30.597878 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03bd73-da98-48bf-90bd-02217147e528 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:30.598252 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03bd73-da98-48bf-90bd-02217147e528 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:30.598602 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc03bd73-da98-48bf-90bd-02217147e528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:30.615699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-cc03bd73-da98-48bf-90bd-02217147e528 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:12:30.616929 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-cc03bd73-da98-48bf-90bd-02217147e528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:30.633557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:12:30.677267 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1425/2859] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:12:30.685629 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e940b-4d8e-43e9-b49e-13156fb816f2 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:30.685950 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e940b-4d8e-43e9-b49e-13156fb816f2 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:30.686232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e940b-4d8e-43e9-b49e-13156fb816f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:30.704806 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-264e940b-4d8e-43e9-b49e-13156fb816f2 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:12:30.706014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-264e940b-4d8e-43e9-b49e-13156fb816f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:30.713840 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1435/2860] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:12:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:12:30.717994 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03bd73-da98-48bf-90bd-02217147e528 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:30.727973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc03bd73-da98-48bf-90bd-02217147e528 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:30.732505 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82fdfd-eed3-4f71-9be2-24c2830ab9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:30.733161 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82fdfd-eed3-4f71-9be2-24c2830ab9e6 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:30.733451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82fdfd-eed3-4f71-9be2-24c2830ab9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:30.733730 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe82fdfd-eed3-4f71-9be2-24c2830ab9e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:30.734459 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82fdfd-eed3-4f71-9be2-24c2830ab9e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:12:30.734768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82fdfd-eed3-4f71-9be2-24c2830ab9e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:30.736880 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe82fdfd-eed3-4f71-9be2-24c2830ab9e6 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:30.748463 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1426/2861] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:12:30.753325 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e940b-4d8e-43e9-b49e-13156fb816f2 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:30.762735 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e940b-4d8e-43e9-b49e-13156fb816f2 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:30.767432 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec0558-898b-4993-bc6f-167c602d5803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:30.768126 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec0558-898b-4993-bc6f-167c602d5803 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:30.768416 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec0558-898b-4993-bc6f-167c602d5803 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:30.768697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec0558-898b-4993-bc6f-167c602d5803 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:30.769424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec0558-898b-4993-bc6f-167c602d5803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:30.771514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec0558-898b-4993-bc6f-167c602d5803 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:30.797181 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1436/2862] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:12:30 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:12:30.801815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82fdfd-eed3-4f71-9be2-24c2830ab9e6 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:30.812002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe82fdfd-eed3-4f71-9be2-24c2830ab9e6 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:30.868938 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b78f5d-e73c-419d-bbf6-c875531d8d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:30.869612 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b78f5d-e73c-419d-bbf6-c875531d8d5b admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:30.869903 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b78f5d-e73c-419d-bbf6-c875531d8d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:30.870186 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b78f5d-e73c-419d-bbf6-c875531d8d5b admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:30.870895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b78f5d-e73c-419d-bbf6-c875531d8d5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:30.872854 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b78f5d-e73c-419d-bbf6-c875531d8d5b admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:30.894301 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1427/2863] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:12:30 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 18:12:30.898360 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec0558-898b-4993-bc6f-167c602d5803 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:30.909106 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec0558-898b-4993-bc6f-167c602d5803 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:30.913755 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea7f7f-7e04-475f-8d92-94506d99d9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:30.914414 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea7f7f-7e04-475f-8d92-94506d99d9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:30.914701 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea7f7f-7e04-475f-8d92-94506d99d9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:30.914983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea7f7f-7e04-475f-8d92-94506d99d9b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:30.915711 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea7f7f-7e04-475f-8d92-94506d99d9b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:30.917341 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea7f7f-7e04-475f-8d92-94506d99d9b9 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:30.928217 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1437/2864] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:12:30.932075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b78f5d-e73c-419d-bbf6-c875531d8d5b admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:30.942882 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b78f5d-e73c-419d-bbf6-c875531d8d5b admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:30.947695 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecc08bc-e4b0-4df1-805d-debdbf73ea78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:30.948387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc08bc-e4b0-4df1-805d-debdbf73ea78 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:30.948682 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc08bc-e4b0-4df1-805d-debdbf73ea78 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:30.948945 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecc08bc-e4b0-4df1-805d-debdbf73ea78 admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:30.949674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc08bc-e4b0-4df1-805d-debdbf73ea78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:30.951254 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecc08bc-e4b0-4df1-805d-debdbf73ea78 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:30.961223 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1428/2865] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:12:30.965439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea7f7f-7e04-475f-8d92-94506d99d9b9 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:30.975841 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea7f7f-7e04-475f-8d92-94506d99d9b9 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:30.980464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089bb8b-90ad-422d-b408-a69aceed1086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:30.981346 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089bb8b-90ad-422d-b408-a69aceed1086 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:30.981630 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089bb8b-90ad-422d-b408-a69aceed1086 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:30.981908 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089bb8b-90ad-422d-b408-a69aceed1086 admin admin] PATH_INFO: `/v3/projects/bc4d39f3674945c88ddcef63926e70b9/users/ed731c050baa4d06950736fa984ac1cc/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:30.996817 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089bb8b-90ad-422d-b408-a69aceed1086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc4d39f3674945c88ddcef63926e70b9, user_id=ed731c050baa4d06950736fa984ac1cc, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:30.999446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089bb8b-90ad-422d-b408-a69aceed1086 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:31.013056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1438/2866] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:12:30 2022] PUT /identity/v3/projects/bc4d39f3674945c88ddcef63926e70b9/users/ed731c050baa4d06950736fa984ac1cc/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:12:31.023559 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c992e7e-0029-4514-b7e5-561d080e43f8 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:31.023874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c992e7e-0029-4514-b7e5-561d080e43f8 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:31.024189 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c992e7e-0029-4514-b7e5-561d080e43f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:31.051444 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-2c992e7e-0029-4514-b7e5-561d080e43f8 None None] MFA Rules not processed for user `ed731c050baa4d06950736fa984ac1cc`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:12:31.052685 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-2c992e7e-0029-4514-b7e5-561d080e43f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:31.115705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1429/2867] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:12:31 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:12:31.127062 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089bb8b-90ad-422d-b408-a69aceed1086 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:31.128810 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8089bb8b-90ad-422d-b408-a69aceed1086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:31.150347 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:12:31.187792 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089bb8b-90ad-422d-b408-a69aceed1086 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:31.191464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66744acd-fb15-4773-bdcd-15c265f8389a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:31.192154 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66744acd-fb15-4773-bdcd-15c265f8389a service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:31.192446 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66744acd-fb15-4773-bdcd-15c265f8389a service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:31.192727 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66744acd-fb15-4773-bdcd-15c265f8389a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:31.200977 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66744acd-fb15-4773-bdcd-15c265f8389a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:31.202394 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66744acd-fb15-4773-bdcd-15c265f8389a service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:31.211162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1439/2868] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:12:31 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:12:31.234904 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c992e7e-0029-4514-b7e5-561d080e43f8 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:31.254035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:12:31.297358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c992e7e-0029-4514-b7e5-561d080e43f8 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:31.302617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb53a5a-bf5a-44f8-bd48-f33b354d332a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:31.303493 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb53a5a-bf5a-44f8-bd48-f33b354d332a admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:31.303776 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb53a5a-bf5a-44f8-bd48-f33b354d332a admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:31.304086 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb53a5a-bf5a-44f8-bd48-f33b354d332a admin admin] PATH_INFO: `/v3/users/ed731c050baa4d06950736fa984ac1cc` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:31.305282 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb53a5a-bf5a-44f8-bd48-f33b354d332a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed731c050baa4d06950736fa984ac1cc)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:31.307184 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb53a5a-bf5a-44f8-bd48-f33b354d332a admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:31.352556 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ddb53a5a-bf5a-44f8-bd48-f33b354d332a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed731c050baa4d06950736fa984ac1cc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:31.364656 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ddb53a5a-bf5a-44f8-bd48-f33b354d332a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed731c050baa4d06950736fa984ac1cc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:31.365340 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ddb53a5a-bf5a-44f8-bd48-f33b354d332a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed731c050baa4d06950736fa984ac1cc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:31.372898 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ddb53a5a-bf5a-44f8-bd48-f33b354d332a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed731c050baa4d06950736fa984ac1cc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:31.373611 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ddb53a5a-bf5a-44f8-bd48-f33b354d332a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed731c050baa4d06950736fa984ac1cc'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:31.385972 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1430/2869] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:12:31 2022] DELETE /identity/v3/users/ed731c050baa4d06950736fa984ac1cc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:12:31.390628 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef3a6b-7155-460c-9482-1ce5187162f6 None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:31.390926 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef3a6b-7155-460c-9482-1ce5187162f6 None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:31.391270 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bef3a6b-7155-460c-9482-1ce5187162f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:31.410038 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-8bef3a6b-7155-460c-9482-1ce5187162f6 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:12:31.411165 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8bef3a6b-7155-460c-9482-1ce5187162f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:31.468732 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1440/2870] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:12:31.479022 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb53a5a-bf5a-44f8-bd48-f33b354d332a admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:31.480421 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ddb53a5a-bf5a-44f8-bd48-f33b354d332a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:31.539594 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb53a5a-bf5a-44f8-bd48-f33b354d332a admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:31.543434 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff2dfa-a4b3-4bb5-92b0-c6d7339d1565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:31.544134 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2dfa-a4b3-4bb5-92b0-c6d7339d1565 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:31.544425 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2dfa-a4b3-4bb5-92b0-c6d7339d1565 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:31.544735 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff2dfa-a4b3-4bb5-92b0-c6d7339d1565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:31.554253 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff2dfa-a4b3-4bb5-92b0-c6d7339d1565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:31.555684 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff2dfa-a4b3-4bb5-92b0-c6d7339d1565 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:31.565172 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1431/2871] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:12:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:12:31.836708 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef3a6b-7155-460c-9482-1ce5187162f6 None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:31.838376 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-8bef3a6b-7155-460c-9482-1ce5187162f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:31.850586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bef3a6b-7155-460c-9482-1ce5187162f6 None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:31.855261 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83e27e-e4b8-4055-8f42-aa5c11444918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:31.856240 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83e27e-e4b8-4055-8f42-aa5c11444918 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:31.856600 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83e27e-e4b8-4055-8f42-aa5c11444918 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:31.856879 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83e27e-e4b8-4055-8f42-aa5c11444918 admin admin] PATH_INFO: `/v3/projects/bc4d39f3674945c88ddcef63926e70b9` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:31.858074 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83e27e-e4b8-4055-8f42-aa5c11444918 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc4d39f3674945c88ddcef63926e70b9)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:31.859997 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83e27e-e4b8-4055-8f42-aa5c11444918 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:31.908523 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-9e83e27e-e4b8-4055-8f42-aa5c11444918 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc4d39f3674945c88ddcef63926e70b9', 'request_id': 'req-9e83e27e-e4b8-4055-8f42-aa5c11444918'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:31.915596 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-9e83e27e-e4b8-4055-8f42-aa5c11444918 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc4d39f3674945c88ddcef63926e70b9', 'request_id': 'req-9e83e27e-e4b8-4055-8f42-aa5c11444918'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:31.921846 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-9e83e27e-e4b8-4055-8f42-aa5c11444918 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc4d39f3674945c88ddcef63926e70b9', 'request_id': 'req-9e83e27e-e4b8-4055-8f42-aa5c11444918'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:31.932187 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-9e83e27e-e4b8-4055-8f42-aa5c11444918 admin admin] The token cache is being invalidate because project bc4d39f3674945c88ddcef63926e70b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:12:31.932479 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-9e83e27e-e4b8-4055-8f42-aa5c11444918 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:31.933115 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-9e83e27e-e4b8-4055-8f42-aa5c11444918 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:31.934646 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1441/2872] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:12:31 2022] DELETE /identity/v3/projects/bc4d39f3674945c88ddcef63926e70b9 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:12:31.940251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4b192-4e58-4f5d-a411-42d97105ac58 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:31.940740 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4b192-4e58-4f5d-a411-42d97105ac58 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:31.940858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce4b192-4e58-4f5d-a411-42d97105ac58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:31.958872 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-5ce4b192-4e58-4f5d-a411-42d97105ac58 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:12:31.960092 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-5ce4b192-4e58-4f5d-a411-42d97105ac58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:31.996841 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:12:32.024733 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1432/2873] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:12:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:12:32.032999 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18322edc-f5cf-40af-b4c7-8ecc07eea9ca None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:32.033297 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18322edc-f5cf-40af-b4c7-8ecc07eea9ca None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:32.033612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18322edc-f5cf-40af-b4c7-8ecc07eea9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:32.051818 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-18322edc-f5cf-40af-b4c7-8ecc07eea9ca None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:12:32.053057 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-18322edc-f5cf-40af-b4c7-8ecc07eea9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:32.060993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1442/2874] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:12:32.065790 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4b192-4e58-4f5d-a411-42d97105ac58 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:32.075217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce4b192-4e58-4f5d-a411-42d97105ac58 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:32.080302 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bf20b-9a59-405d-98c3-607a0c3478ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:32.080944 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bf20b-9a59-405d-98c3-607a0c3478ff admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:32.081217 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bf20b-9a59-405d-98c3-607a0c3478ff admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:32.081495 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127bf20b-9a59-405d-98c3-607a0c3478ff admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:32.082213 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bf20b-9a59-405d-98c3-607a0c3478ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:12:32.082515 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bf20b-9a59-405d-98c3-607a0c3478ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:32.084136 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127bf20b-9a59-405d-98c3-607a0c3478ff admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:32.095659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1433/2875] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:12:32.100195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18322edc-f5cf-40af-b4c7-8ecc07eea9ca None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:32.109750 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18322edc-f5cf-40af-b4c7-8ecc07eea9ca None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:32.114512 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ccdd5-3253-4959-b14c-9667c50c91d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:32.115195 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ccdd5-3253-4959-b14c-9667c50c91d4 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:32.116173 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ccdd5-3253-4959-b14c-9667c50c91d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:32.116489 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ccdd5-3253-4959-b14c-9667c50c91d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:32.117222 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ccdd5-3253-4959-b14c-9667c50c91d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:32.119143 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218ccdd5-3253-4959-b14c-9667c50c91d4 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:32.148570 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1443/2876] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:12:32 2022] POST /identity/v3/projects => generated 399 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:12:32.150047 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bf20b-9a59-405d-98c3-607a0c3478ff admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:32.160617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127bf20b-9a59-405d-98c3-607a0c3478ff admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:32.216659 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50199c46-e106-4f55-aeb7-dca1069fb6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:32.217251 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50199c46-e106-4f55-aeb7-dca1069fb6b4 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:32.217533 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50199c46-e106-4f55-aeb7-dca1069fb6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:32.217812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50199c46-e106-4f55-aeb7-dca1069fb6b4 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:32.218494 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50199c46-e106-4f55-aeb7-dca1069fb6b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:32.220462 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50199c46-e106-4f55-aeb7-dca1069fb6b4 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:32.240609 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1434/2877] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:12:32 2022] POST /identity/v3/users => generated 300 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:12:32.244742 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ccdd5-3253-4959-b14c-9667c50c91d4 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:32.256169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ccdd5-3253-4959-b14c-9667c50c91d4 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:32.260664 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d614e-3614-4991-965e-6c1ba877de9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:32.261323 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d614e-3614-4991-965e-6c1ba877de9b admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:32.261605 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d614e-3614-4991-965e-6c1ba877de9b admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:32.261882 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d614e-3614-4991-965e-6c1ba877de9b admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:32.262602 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d614e-3614-4991-965e-6c1ba877de9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:32.264215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d614e-3614-4991-965e-6c1ba877de9b admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:32.274531 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1444/2878] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:12:32.278738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50199c46-e106-4f55-aeb7-dca1069fb6b4 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:32.288630 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50199c46-e106-4f55-aeb7-dca1069fb6b4 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:32.293123 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358f0d6d-8f25-44f8-8345-5b99400e377c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:32.293773 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f0d6d-8f25-44f8-8345-5b99400e377c admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:32.294063 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f0d6d-8f25-44f8-8345-5b99400e377c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:32.294356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358f0d6d-8f25-44f8-8345-5b99400e377c admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:32.295397 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f0d6d-8f25-44f8-8345-5b99400e377c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:32.297050 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358f0d6d-8f25-44f8-8345-5b99400e377c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:32.307436 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1435/2879] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:12:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 18:12:32.311864 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d614e-3614-4991-965e-6c1ba877de9b admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:32.322676 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d614e-3614-4991-965e-6c1ba877de9b admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:32.327485 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7171fa-0e7b-4f5f-8690-f187022a8bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:32.328417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7171fa-0e7b-4f5f-8690-f187022a8bf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:32.328707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7171fa-0e7b-4f5f-8690-f187022a8bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:32.328989 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7171fa-0e7b-4f5f-8690-f187022a8bf9 admin admin] PATH_INFO: `/v3/projects/a2372fc177924aa29abfbce58be48aab/users/0799ed4bbfb94c2facf3400cb3970fbb/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:32.331393 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:12:32.347903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7171fa-0e7b-4f5f-8690-f187022a8bf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2372fc177924aa29abfbce58be48aab, user_id=0799ed4bbfb94c2facf3400cb3970fbb, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:32.350961 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7171fa-0e7b-4f5f-8690-f187022a8bf9 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:32.364439 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1445/2880] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:12:32 2022] PUT /identity/v3/projects/a2372fc177924aa29abfbce58be48aab/users/0799ed4bbfb94c2facf3400cb3970fbb/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:12:32.375361 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac4022-32d8-422c-9ba0-bffe9430d427 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:32.375657 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac4022-32d8-422c-9ba0-bffe9430d427 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:32.375971 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac4022-32d8-422c-9ba0-bffe9430d427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:32.403988 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-a4ac4022-32d8-422c-9ba0-bffe9430d427 None None] MFA Rules not processed for user `0799ed4bbfb94c2facf3400cb3970fbb`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:12:32.405229 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-a4ac4022-32d8-422c-9ba0-bffe9430d427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:32.471147 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1436/2881] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:12:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:12:32.482433 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7171fa-0e7b-4f5f-8690-f187022a8bf9 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:32.484123 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-3f7171fa-0e7b-4f5f-8690-f187022a8bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:32.541594 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7171fa-0e7b-4f5f-8690-f187022a8bf9 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:32.545116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738a8599-0159-4fa8-b6d8-766bd80d4e21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:32.545760 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a8599-0159-4fa8-b6d8-766bd80d4e21 service cinder] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:32.546048 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a8599-0159-4fa8-b6d8-766bd80d4e21 service cinder] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:32.546324 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738a8599-0159-4fa8-b6d8-766bd80d4e21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:32.555171 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a8599-0159-4fa8-b6d8-766bd80d4e21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:32.556642 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738a8599-0159-4fa8-b6d8-766bd80d4e21 service cinder] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:32.565747 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1446/2882] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:12:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:12:40.601053 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac4022-32d8-422c-9ba0-bffe9430d427 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:40.639557 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:12:40.663527 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac4022-32d8-422c-9ba0-bffe9430d427 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:40.668119 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5608aaf-2200-4073-b9da-87b8725ecc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:40.668986 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5608aaf-2200-4073-b9da-87b8725ecc05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:40.669270 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5608aaf-2200-4073-b9da-87b8725ecc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:40.669547 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5608aaf-2200-4073-b9da-87b8725ecc05 admin admin] PATH_INFO: `/v3/users/0799ed4bbfb94c2facf3400cb3970fbb` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:40.670737 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5608aaf-2200-4073-b9da-87b8725ecc05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0799ed4bbfb94c2facf3400cb3970fbb)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:40.673334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5608aaf-2200-4073-b9da-87b8725ecc05 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:40.717699 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c5608aaf-2200-4073-b9da-87b8725ecc05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0799ed4bbfb94c2facf3400cb3970fbb'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:40.729393 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c5608aaf-2200-4073-b9da-87b8725ecc05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0799ed4bbfb94c2facf3400cb3970fbb'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:40.730061 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c5608aaf-2200-4073-b9da-87b8725ecc05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0799ed4bbfb94c2facf3400cb3970fbb'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:40.737576 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c5608aaf-2200-4073-b9da-87b8725ecc05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0799ed4bbfb94c2facf3400cb3970fbb'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:40.738266 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-c5608aaf-2200-4073-b9da-87b8725ecc05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0799ed4bbfb94c2facf3400cb3970fbb'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:40.750427 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1437/2883] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:12:40 2022] DELETE /identity/v3/users/0799ed4bbfb94c2facf3400cb3970fbb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:12:40.755484 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86bd25-d745-4bf3-a39e-3aa96dc8722e None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:40.755995 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86bd25-d745-4bf3-a39e-3aa96dc8722e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:40.756215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86bd25-d745-4bf3-a39e-3aa96dc8722e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:40.773136 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-4c86bd25-d745-4bf3-a39e-3aa96dc8722e None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:12:40.774353 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4c86bd25-d745-4bf3-a39e-3aa96dc8722e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:40.833387 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1447/2884] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:12:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:12:40.842997 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5608aaf-2200-4073-b9da-87b8725ecc05 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:40.844987 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c5608aaf-2200-4073-b9da-87b8725ecc05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:40.900915 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5608aaf-2200-4073-b9da-87b8725ecc05 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:40.904828 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e14c50-71de-497f-a399-943b928ab7b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:40.905472 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e14c50-71de-497f-a399-943b928ab7b6 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:40.905757 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e14c50-71de-497f-a399-943b928ab7b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:40.906035 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e14c50-71de-497f-a399-943b928ab7b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:40.915479 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e14c50-71de-497f-a399-943b928ab7b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:40.916927 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e14c50-71de-497f-a399-943b928ab7b6 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:40.926877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1438/2885] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:12:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:12:41.193131 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86bd25-d745-4bf3-a39e-3aa96dc8722e None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:41.194697 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-4c86bd25-d745-4bf3-a39e-3aa96dc8722e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:41.209502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86bd25-d745-4bf3-a39e-3aa96dc8722e None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:41.214821 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:41.215892 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:41.216267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:41.216482 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1 admin admin] PATH_INFO: `/v3/projects/a2372fc177924aa29abfbce58be48aab` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:41.217699 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2372fc177924aa29abfbce58be48aab)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:41.220263 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:41.266668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2372fc177924aa29abfbce58be48aab', 'request_id': 'req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:41.273575 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2372fc177924aa29abfbce58be48aab', 'request_id': 'req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:41.280248 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2372fc177924aa29abfbce58be48aab', 'request_id': 'req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:41.291365 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1 admin admin] The token cache is being invalidate because project a2372fc177924aa29abfbce58be48aab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91926) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:12:41.291654 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:41.292329 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-81a238a5-41f6-45fb-80f9-9f0b7f3176c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:12:41.293888 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1448/2886] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:12:41 2022] DELETE /identity/v3/projects/a2372fc177924aa29abfbce58be48aab => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:12:41.300048 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167a6c9-2214-4b74-90ab-28edaf1c5a69 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:41.300344 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167a6c9-2214-4b74-90ab-28edaf1c5a69 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:41.300655 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7167a6c9-2214-4b74-90ab-28edaf1c5a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:41.317959 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-7167a6c9-2214-4b74-90ab-28edaf1c5a69 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:12:41.319196 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7167a6c9-2214-4b74-90ab-28edaf1c5a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:41.369623 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:12:41.379635 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1439/2887] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:12:41.387347 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978802b-6b49-42d4-86b0-109dfc55db4d None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:41.387640 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978802b-6b49-42d4-86b0-109dfc55db4d None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:41.387919 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978802b-6b49-42d4-86b0-109dfc55db4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:41.407574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-7978802b-6b49-42d4-86b0-109dfc55db4d None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:12:41.408827 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-7978802b-6b49-42d4-86b0-109dfc55db4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:41.416780 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1449/2888] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:12:41.420994 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167a6c9-2214-4b74-90ab-28edaf1c5a69 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:41.430558 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7167a6c9-2214-4b74-90ab-28edaf1c5a69 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:41.435137 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916fea4-987b-48a4-8203-0331f6d455d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:41.435782 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916fea4-987b-48a4-8203-0331f6d455d4 admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:41.436106 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916fea4-987b-48a4-8203-0331f6d455d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:41.436387 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2916fea4-987b-48a4-8203-0331f6d455d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:41.437118 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916fea4-987b-48a4-8203-0331f6d455d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=91925) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 18:12:41.437429 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916fea4-987b-48a4-8203-0331f6d455d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:41.439026 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2916fea4-987b-48a4-8203-0331f6d455d4 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:41.450394 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1440/2889] 10.222.0.64 () {58 vars in 1152 bytes} [Thu Jul 14 18:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 18:12:41.455131 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978802b-6b49-42d4-86b0-109dfc55db4d None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:41.459651 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:12:41.465871 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978802b-6b49-42d4-86b0-109dfc55db4d None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:41.472068 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842efec6-4b9d-4026-8d36-a4a86e8a03e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:41.472734 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842efec6-4b9d-4026-8d36-a4a86e8a03e1 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:41.473021 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842efec6-4b9d-4026-8d36-a4a86e8a03e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:41.473301 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842efec6-4b9d-4026-8d36-a4a86e8a03e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:41.474001 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842efec6-4b9d-4026-8d36-a4a86e8a03e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:41.476000 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842efec6-4b9d-4026-8d36-a4a86e8a03e1 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:41.501773 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1450/2890] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:12:41 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:12:41.505442 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916fea4-987b-48a4-8203-0331f6d455d4 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:41.515578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2916fea4-987b-48a4-8203-0331f6d455d4 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:41.573777 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd26b0-5904-4420-8326-695807f3c493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:41.574451 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd26b0-5904-4420-8326-695807f3c493 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:41.574732 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd26b0-5904-4420-8326-695807f3c493 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:41.575009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd26b0-5904-4420-8326-695807f3c493 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:41.575694 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd26b0-5904-4420-8326-695807f3c493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:41.577635 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd26b0-5904-4420-8326-695807f3c493 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:41.597408 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1441/2891] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:12:41 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:12:41.601693 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842efec6-4b9d-4026-8d36-a4a86e8a03e1 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:41.612171 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842efec6-4b9d-4026-8d36-a4a86e8a03e1 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:41.616766 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4e899-d80f-4dc6-90e2-155a859e7e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:41.617430 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4e899-d80f-4dc6-90e2-155a859e7e5c admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:41.617720 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4e899-d80f-4dc6-90e2-155a859e7e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:41.618005 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4e899-d80f-4dc6-90e2-155a859e7e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:41.618739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4e899-d80f-4dc6-90e2-155a859e7e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:41.620430 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4e899-d80f-4dc6-90e2-155a859e7e5c admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:41.631178 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1451/2892] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:12:41.635097 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd26b0-5904-4420-8326-695807f3c493 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:41.645932 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd26b0-5904-4420-8326-695807f3c493 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:41.650629 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0a532-d114-42f7-b86e-1b9fb75f331c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:41.651288 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0a532-d114-42f7-b86e-1b9fb75f331c admin admin] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:41.651578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0a532-d114-42f7-b86e-1b9fb75f331c admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:41.651858 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0a532-d114-42f7-b86e-1b9fb75f331c admin admin] PATH_INFO: `/v3/roles` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:41.652874 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0a532-d114-42f7-b86e-1b9fb75f331c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:41.654508 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0a532-d114-42f7-b86e-1b9fb75f331c admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:41.664488 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1442/2893] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:12:41.668593 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4e899-d80f-4dc6-90e2-155a859e7e5c admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:41.678514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4e899-d80f-4dc6-90e2-155a859e7e5c admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:41.683287 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5d50d-fa8c-4591-90d8-475ac2322cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:41.684289 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5d50d-fa8c-4591-90d8-475ac2322cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:41.684486 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5d50d-fa8c-4591-90d8-475ac2322cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:41.684765 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf5d50d-fa8c-4591-90d8-475ac2322cd9 admin admin] PATH_INFO: `/v3/projects/3a189153b8f44520a5bac2c4bcedc013/users/15d9d47d160b4595a16bf23eaa509a08/roles/fbd0f348273f4791b1813d1df24df10e` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:41.699548 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5d50d-fa8c-4591-90d8-475ac2322cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a189153b8f44520a5bac2c4bcedc013, user_id=15d9d47d160b4595a16bf23eaa509a08, role_id=fbd0f348273f4791b1813d1df24df10e)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:41.702259 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf5d50d-fa8c-4591-90d8-475ac2322cd9 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:41.716941 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1452/2894] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:12:41 2022] PUT /identity/v3/projects/3a189153b8f44520a5bac2c4bcedc013/users/15d9d47d160b4595a16bf23eaa509a08/roles/fbd0f348273f4791b1813d1df24df10e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:12:41.722075 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9f4fb-4dd7-4cec-8a43-6ac7f0721372 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:41.722367 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9f4fb-4dd7-4cec-8a43-6ac7f0721372 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:41.722674 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa9f4fb-4dd7-4cec-8a43-6ac7f0721372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:41.740658 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-bfa9f4fb-4dd7-4cec-8a43-6ac7f0721372 None None] MFA Rules not processed for user `7d77dedfd6bd435d94f994cfcde5100a`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:12:41.741942 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-bfa9f4fb-4dd7-4cec-8a43-6ac7f0721372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:41.800712 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1443/2895] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:12:41.810618 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5d50d-fa8c-4591-90d8-475ac2322cd9 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:41.812574 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-fcf5d50d-fa8c-4591-90d8-475ac2322cd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:41.870404 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf5d50d-fa8c-4591-90d8-475ac2322cd9 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:41.874246 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55963a8b-1a02-4a2f-9ae4-df519c8f588c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:41.874849 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55963a8b-1a02-4a2f-9ae4-df519c8f588c service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:41.875124 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55963a8b-1a02-4a2f-9ae4-df519c8f588c service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:41.875955 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55963a8b-1a02-4a2f-9ae4-df519c8f588c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:41.885201 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55963a8b-1a02-4a2f-9ae4-df519c8f588c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:41.886629 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55963a8b-1a02-4a2f-9ae4-df519c8f588c service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:41.896679 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1453/2896] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:12:45.794791 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43ef23-fe8a-4577-bb46-9b264749cf84 None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:45.795319 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43ef23-fe8a-4577-bb46-9b264749cf84 None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:45.795447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43ef23-fe8a-4577-bb46-9b264749cf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:45.825582 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-ed43ef23-fe8a-4577-bb46-9b264749cf84 None None] MFA Rules not processed for user `15d9d47d160b4595a16bf23eaa509a08`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:12:45.826824 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ed43ef23-fe8a-4577-bb46-9b264749cf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:45.890303 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:12:45.901985 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1444/2897] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:12:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:12:45.906991 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55963a8b-1a02-4a2f-9ae4-df519c8f588c service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:45.917026 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55963a8b-1a02-4a2f-9ae4-df519c8f588c service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:45.922464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abffa9b-2804-44aa-8cf4-ed90c9f61282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:45.923362 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abffa9b-2804-44aa-8cf4-ed90c9f61282 admin admin] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:45.923647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abffa9b-2804-44aa-8cf4-ed90c9f61282 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:45.923950 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abffa9b-2804-44aa-8cf4-ed90c9f61282 admin admin] PATH_INFO: `/v3/projects` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:45.924668 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abffa9b-2804-44aa-8cf4-ed90c9f61282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:45.926647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abffa9b-2804-44aa-8cf4-ed90c9f61282 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:45.955092 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1454/2898] 10.222.0.64 () {60 vars in 1152 bytes} [Thu Jul 14 18:12:45 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:12:45.958938 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43ef23-fe8a-4577-bb46-9b264749cf84 None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:45.971410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43ef23-fe8a-4577-bb46-9b264749cf84 None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:46.032790 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5cf82-9928-4477-8ac7-41a6e7d9b8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:46.033756 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5cf82-9928-4477-8ac7-41a6e7d9b8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:46.034046 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5cf82-9928-4477-8ac7-41a6e7d9b8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:46.034328 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5cf82-9928-4477-8ac7-41a6e7d9b8b1 admin admin] PATH_INFO: `/v3/users` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:46.035039 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5cf82-9928-4477-8ac7-41a6e7d9b8b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:46.037057 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5cf82-9928-4477-8ac7-41a6e7d9b8b1 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:46.059762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1445/2899] 10.222.0.64 () {60 vars in 1143 bytes} [Thu Jul 14 18:12:45 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 18:12:46.063885 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abffa9b-2804-44aa-8cf4-ed90c9f61282 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:46.075005 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abffa9b-2804-44aa-8cf4-ed90c9f61282 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:46.079647 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e5de6-6be2-49c9-b7fd-5d7f713c9e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:46.080417 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e5de6-6be2-49c9-b7fd-5d7f713c9e05 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:46.080702 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e5de6-6be2-49c9-b7fd-5d7f713c9e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:46.080980 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e5de6-6be2-49c9-b7fd-5d7f713c9e05 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:46.081707 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e5de6-6be2-49c9-b7fd-5d7f713c9e05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:46.083311 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256e5de6-6be2-49c9-b7fd-5d7f713c9e05 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:46.095636 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1455/2900] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:12:46.100201 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5cf82-9928-4477-8ac7-41a6e7d9b8b1 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:46.116524 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5cf82-9928-4477-8ac7-41a6e7d9b8b1 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:46.124616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321833df-a4ab-4284-a382-8be3b5e2197d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:46.125715 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321833df-a4ab-4284-a382-8be3b5e2197d admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:46.126001 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321833df-a4ab-4284-a382-8be3b5e2197d admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:46.126465 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321833df-a4ab-4284-a382-8be3b5e2197d admin admin] PATH_INFO: `/v3/projects/8ff43d2a08634ce4aa7a214ea2951f98/users/8d639a61029d49a78164037e89adfc41/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:46.142653 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321833df-a4ab-4284-a382-8be3b5e2197d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff43d2a08634ce4aa7a214ea2951f98, user_id=8d639a61029d49a78164037e89adfc41, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:46.145290 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321833df-a4ab-4284-a382-8be3b5e2197d admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:46.160646 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1446/2901] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:12:46 2022] PUT /identity/v3/projects/8ff43d2a08634ce4aa7a214ea2951f98/users/8d639a61029d49a78164037e89adfc41/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:12:46.165726 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e5de6-6be2-49c9-b7fd-5d7f713c9e05 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:46.177389 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e5de6-6be2-49c9-b7fd-5d7f713c9e05 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:46.237519 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d3b581-2b97-4ac8-a458-bd312d73c6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:46.238225 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3b581-2b97-4ac8-a458-bd312d73c6c0 admin admin] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:46.238518 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3b581-2b97-4ac8-a458-bd312d73c6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:46.238795 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d3b581-2b97-4ac8-a458-bd312d73c6c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:46.239557 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3b581-2b97-4ac8-a458-bd312d73c6c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:46.241215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d3b581-2b97-4ac8-a458-bd312d73c6c0 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:46.251543 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1456/2902] 10.222.0.64 () {58 vars in 1121 bytes} [Thu Jul 14 18:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 18:12:46.255877 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321833df-a4ab-4284-a382-8be3b5e2197d admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:46.268812 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321833df-a4ab-4284-a382-8be3b5e2197d admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:46.273519 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1f6de-3849-47ec-8cfc-e9d4317a8910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:46.274416 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1f6de-3849-47ec-8cfc-e9d4317a8910 admin admin] REQUEST_METHOD: `PUT` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:46.274698 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1f6de-3849-47ec-8cfc-e9d4317a8910 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:46.274973 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1f6de-3849-47ec-8cfc-e9d4317a8910 admin admin] PATH_INFO: `/v3/projects/8ff43d2a08634ce4aa7a214ea2951f98/users/8d639a61029d49a78164037e89adfc41/roles/c546ef95fd8241208d9fc75ab478ff7d` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:46.277095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1f6de-3849-47ec-8cfc-e9d4317a8910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ff43d2a08634ce4aa7a214ea2951f98, user_id=8d639a61029d49a78164037e89adfc41, role_id=c546ef95fd8241208d9fc75ab478ff7d)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:46.280029 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1f6de-3849-47ec-8cfc-e9d4317a8910 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:46.291713 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1447/2903] 10.222.0.64 () {60 vars in 1482 bytes} [Thu Jul 14 18:12:46 2022] PUT /identity/v3/projects/8ff43d2a08634ce4aa7a214ea2951f98/users/8d639a61029d49a78164037e89adfc41/roles/c546ef95fd8241208d9fc75ab478ff7d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:12:50.833942 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd78962-fb4e-46ee-bcbf-fd4fa8e1737e None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:50.835252 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd78962-fb4e-46ee-bcbf-fd4fa8e1737e None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:50.835539 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd78962-fb4e-46ee-bcbf-fd4fa8e1737e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:50.867862 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:12:50.877964 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-bdd78962-fb4e-46ee-bcbf-fd4fa8e1737e None None] MFA Rules not processed for user `8d639a61029d49a78164037e89adfc41`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:12:50.880606 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-bdd78962-fb4e-46ee-bcbf-fd4fa8e1737e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:50.975976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1457/2904] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:12:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:12:50.989568 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1f6de-3849-47ec-8cfc-e9d4317a8910 admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:12:50.992024 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-30f1f6de-3849-47ec-8cfc-e9d4317a8910 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:12:51.097298 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1f6de-3849-47ec-8cfc-e9d4317a8910 admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:12:51.101223 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398ec30-94d8-4667-bfe4-6bd556ad287f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:12:51.102004 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398ec30-94d8-4667-bfe4-6bd556ad287f service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:12:51.102355 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398ec30-94d8-4667-bfe4-6bd556ad287f service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:12:51.102705 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398ec30-94d8-4667-bfe4-6bd556ad287f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:12:51.154401 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:12:51.170941 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398ec30-94d8-4667-bfe4-6bd556ad287f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:12:51.173005 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398ec30-94d8-4667-bfe4-6bd556ad287f service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:12:51.182792 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1448/2905] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:12:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:13:00.963586 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=29 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:13:01.178173 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=8 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:13:14.991913 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7336c-fb55-45c0-bc8b-197dfc2537ba None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:13:14.992482 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7336c-fb55-45c0-bc8b-197dfc2537ba None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:13:14.992590 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f7336c-fb55-45c0-bc8b-197dfc2537ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:13:15.010176 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-a4f7336c-fb55-45c0-bc8b-197dfc2537ba None None] MFA Rules not processed for user `15d9d47d160b4595a16bf23eaa509a08`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:13:15.011364 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-a4f7336c-fb55-45c0-bc8b-197dfc2537ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:13:15.019027 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1458/2906] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:13:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:13:15.059976 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398ec30-94d8-4667-bfe4-6bd556ad287f service nova] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:13:15.061395 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-8398ec30-94d8-4667-bfe4-6bd556ad287f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:13:15.117792 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398ec30-94d8-4667-bfe4-6bd556ad287f service nova] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:13:15.121768 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ef279-5654-48bf-bbb6-c2cb1aa44759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:13:15.122039 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ef279-5654-48bf-bbb6-c2cb1aa44759 service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:13:15.122323 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ef279-5654-48bf-bbb6-c2cb1aa44759 service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:13:15.122601 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ef279-5654-48bf-bbb6-c2cb1aa44759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:13:15.124410 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-fb9ef279-5654-48bf-bbb6-c2cb1aa44759 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:13:15.137803 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ef279-5654-48bf-bbb6-c2cb1aa44759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:13:15.139277 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ef279-5654-48bf-bbb6-c2cb1aa44759 service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:13:15.151044 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1449/2907] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:13:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:13:15.604876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7336c-fb55-45c0-bc8b-197dfc2537ba None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:13:15.660585 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f7336c-fb55-45c0-bc8b-197dfc2537ba None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:13:15.664234 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c797c-1af7-4bfd-878c-03d56b2753a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:13:15.664834 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c797c-1af7-4bfd-878c-03d56b2753a7 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:13:15.665119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c797c-1af7-4bfd-878c-03d56b2753a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:13:15.665397 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c797c-1af7-4bfd-878c-03d56b2753a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:13:15.667190 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1a5c797c-1af7-4bfd-878c-03d56b2753a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:13:15.677129 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c797c-1af7-4bfd-878c-03d56b2753a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:13:15.678542 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c797c-1af7-4bfd-878c-03d56b2753a7 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:13:15.686488 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1459/2908] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:13:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:13:16.134934 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ef279-5654-48bf-bbb6-c2cb1aa44759 service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:13:16.145109 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ef279-5654-48bf-bbb6-c2cb1aa44759 service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:13:16.148784 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1adab1-a34f-45ec-b6fe-716ef9611787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:13:16.149586 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1adab1-a34f-45ec-b6fe-716ef9611787 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:13:16.149961 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1adab1-a34f-45ec-b6fe-716ef9611787 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:13:16.150522 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1adab1-a34f-45ec-b6fe-716ef9611787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:13:16.152370 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-7f1adab1-a34f-45ec-b6fe-716ef9611787 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:13:16.168560 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1adab1-a34f-45ec-b6fe-716ef9611787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:13:16.170081 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1adab1-a34f-45ec-b6fe-716ef9611787 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:13:16.182641 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1450/2909] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:13:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 18:13:17.815779 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c797c-1af7-4bfd-878c-03d56b2753a7 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:13:17.817416 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-1a5c797c-1af7-4bfd-878c-03d56b2753a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:13:17.828837 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c797c-1af7-4bfd-878c-03d56b2753a7 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:13:17.833049 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358dc89-6a7e-4dd3-9184-99a44413f4c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:13:17.833709 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358dc89-6a7e-4dd3-9184-99a44413f4c7 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:13:17.833990 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358dc89-6a7e-4dd3-9184-99a44413f4c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:13:17.834266 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358dc89-6a7e-4dd3-9184-99a44413f4c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:13:17.836215 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e358dc89-6a7e-4dd3-9184-99a44413f4c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:13:17.847309 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358dc89-6a7e-4dd3-9184-99a44413f4c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:13:17.848974 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358dc89-6a7e-4dd3-9184-99a44413f4c7 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:13:17.859243 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1460/2910] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:13:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:13:26.176738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=32 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:13:27.854815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=35 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:13:31.542482 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1adab1-a34f-45ec-b6fe-716ef9611787 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:13:31.551728 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1adab1-a34f-45ec-b6fe-716ef9611787 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:13:31.555372 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf259f-88df-41d0-b614-36529aa7af99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:13:31.556066 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf259f-88df-41d0-b614-36529aa7af99 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:13:31.556356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf259f-88df-41d0-b614-36529aa7af99 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:13:31.556635 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf259f-88df-41d0-b614-36529aa7af99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:13:31.566164 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf259f-88df-41d0-b614-36529aa7af99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:13:31.567587 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cf259f-88df-41d0-b614-36529aa7af99 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:13:31.577815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1451/2911] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:13:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:13:33.128578 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358dc89-6a7e-4dd3-9184-99a44413f4c7 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:13:33.130106 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-e358dc89-6a7e-4dd3-9184-99a44413f4c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:13:33.147119 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358dc89-6a7e-4dd3-9184-99a44413f4c7 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:13:33.153139 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac05ddb-ee61-4c86-b27a-8085e2262dc3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:13:33.153893 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac05ddb-ee61-4c86-b27a-8085e2262dc3 service nova] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:13:33.154255 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac05ddb-ee61-4c86-b27a-8085e2262dc3 service nova] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:13:33.154610 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac05ddb-ee61-4c86-b27a-8085e2262dc3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:13:33.157121 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-eac05ddb-ee61-4c86-b27a-8085e2262dc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:13:33.169559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac05ddb-ee61-4c86-b27a-8085e2262dc3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:13:33.171157 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac05ddb-ee61-4c86-b27a-8085e2262dc3 service nova] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:13:33.182184 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1461/2912] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:13:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:13:41.573702 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=6 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:13:43.177559 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=6 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:15:56.445863 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb69bb-14a8-4fa5-b62b-b1cfce199c4e None None] REQUEST_METHOD: `POST` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:15:56.446358 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb69bb-14a8-4fa5-b62b-b1cfce199c4e None None] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:15:56.446475 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb69bb-14a8-4fa5-b62b-b1cfce199c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:15:56.463934 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.auth.core [None req-4cdb69bb-14a8-4fa5-b62b-b1cfce199c4e None None] MFA Rules not processed for user `15d9d47d160b4595a16bf23eaa509a08`. Rule list: `[]` (Enabled: `True`). {{(pid=91925) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:15:56.465169 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-4cdb69bb-14a8-4fa5-b62b-b1cfce199c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:15:56.472153 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1452/2913] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:15:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:15:57.220495 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac05ddb-ee61-4c86-b27a-8085e2262dc3 service nova] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:15:57.221845 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-eac05ddb-ee61-4c86-b27a-8085e2262dc3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:15:57.293405 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac05ddb-ee61-4c86-b27a-8085e2262dc3 service nova] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:15:57.299097 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f940a-4e51-4863-8b9e-5d2b1e9daf5e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:15:57.299660 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f940a-4e51-4863-8b9e-5d2b1e9daf5e service placement] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:15:57.299976 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f940a-4e51-4863-8b9e-5d2b1e9daf5e service placement] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:15:57.300262 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f940a-4e51-4863-8b9e-5d2b1e9daf5e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:15:57.302050 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-596f940a-4e51-4863-8b9e-5d2b1e9daf5e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:15:57.313826 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f940a-4e51-4863-8b9e-5d2b1e9daf5e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:15:57.315437 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f940a-4e51-4863-8b9e-5d2b1e9daf5e service placement] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:15:57.325632 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1462/2914] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:15:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:16:06.459212 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=3 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:16:07.321283 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=32 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:16:11.950497 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb69bb-14a8-4fa5-b62b-b1cfce199c4e None None] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:16:11.965190 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb69bb-14a8-4fa5-b62b-b1cfce199c4e None None] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:16:11.971191 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60ba46-c498-41de-afba-538b2e5bcc85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:16:11.971838 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60ba46-c498-41de-afba-538b2e5bcc85 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:16:11.972152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60ba46-c498-41de-afba-538b2e5bcc85 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:16:11.972430 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60ba46-c498-41de-afba-538b2e5bcc85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:16:11.974198 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ff60ba46-c498-41de-afba-538b2e5bcc85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:16:11.987861 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60ba46-c498-41de-afba-538b2e5bcc85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:16:11.989464 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60ba46-c498-41de-afba-538b2e5bcc85 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:16:12.000519 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1453/2915] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:16:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:16:15.102612 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f940a-4e51-4863-8b9e-5d2b1e9daf5e service placement] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:16:15.113477 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f940a-4e51-4863-8b9e-5d2b1e9daf5e service placement] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:16:15.118759 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c464e-135e-4e1e-8fba-cb77ac11db58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:16:15.119427 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c464e-135e-4e1e-8fba-cb77ac11db58 service placement] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:16:15.119841 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c464e-135e-4e1e-8fba-cb77ac11db58 service placement] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:16:15.120164 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c464e-135e-4e1e-8fba-cb77ac11db58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:16:15.121981 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ac0c464e-135e-4e1e-8fba-cb77ac11db58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:16:15.134800 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c464e-135e-4e1e-8fba-cb77ac11db58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:16:15.136328 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c464e-135e-4e1e-8fba-cb77ac11db58 service placement] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:16:15.147420 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1463/2916] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:16:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:16:21.995938 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=6 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:16:25.142825 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=6 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:18:05.583998 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60ba46-c498-41de-afba-538b2e5bcc85 service nova] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:18:05.585390 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ff60ba46-c498-41de-afba-538b2e5bcc85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:18:05.597500 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60ba46-c498-41de-afba-538b2e5bcc85 service nova] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:18:05.601068 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bffbee-5c14-4566-acc4-4a26f58772ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:18:05.601719 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bffbee-5c14-4566-acc4-4a26f58772ba service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:18:05.602002 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bffbee-5c14-4566-acc4-4a26f58772ba service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:18:05.602280 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bffbee-5c14-4566-acc4-4a26f58772ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:18:05.610484 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bffbee-5c14-4566-acc4-4a26f58772ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:18:05.611895 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bffbee-5c14-4566-acc4-4a26f58772ba service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:18:05.620653 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1454/2917] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:18:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:18:15.616743 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=6 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:18:17.638861 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ec018-046f-4d17-8b89-d2d5999b4dbb None None] REQUEST_METHOD: `POST` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:18:17.639426 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ec018-046f-4d17-8b89-d2d5999b4dbb None None] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:18:17.639426 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ec018-046f-4d17-8b89-d2d5999b4dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:18:17.656428 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.auth.core [None req-b31ec018-046f-4d17-8b89-d2d5999b4dbb None None] MFA Rules not processed for user `15d9d47d160b4595a16bf23eaa509a08`. Rule list: `[]` (Enabled: `True`). {{(pid=91926) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 18:18:17.657669 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b31ec018-046f-4d17-8b89-d2d5999b4dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:18:17.665056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1464/2918] 10.222.0.64 () {58 vars in 957 bytes} [Thu Jul 14 18:18:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 18:18:17.703891 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bffbee-5c14-4566-acc4-4a26f58772ba service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:18:17.712565 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bffbee-5c14-4566-acc4-4a26f58772ba service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:18:17.716152 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc8f6a-82bc-419d-82e8-20195efcf2fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e49fad7db3c643b99e9071d9ca5f0196', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:18:17.716793 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc8f6a-82bc-419d-82e8-20195efcf2fb service cinder] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:18:17.717078 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc8f6a-82bc-419d-82e8-20195efcf2fb service cinder] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:18:17.717356 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bc8f6a-82bc-419d-82e8-20195efcf2fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:18:17.728160 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc8f6a-82bc-419d-82e8-20195efcf2fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:18:17.729584 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bc8f6a-82bc-419d-82e8-20195efcf2fb service cinder] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:18:17.741620 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1455/2919] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:18:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:18:18.027280 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ec018-046f-4d17-8b89-d2d5999b4dbb None None] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:18:18.037686 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ec018-046f-4d17-8b89-d2d5999b4dbb None None] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:18:18.041499 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af356c3-60f7-45a8-8d5a-3cbb6c64baf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:18:18.042150 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af356c3-60f7-45a8-8d5a-3cbb6c64baf6 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:18:18.042464 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af356c3-60f7-45a8-8d5a-3cbb6c64baf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:18:18.042745 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af356c3-60f7-45a8-8d5a-3cbb6c64baf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:18:18.044527 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-5af356c3-60f7-45a8-8d5a-3cbb6c64baf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:18:18.054514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af356c3-60f7-45a8-8d5a-3cbb6c64baf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:18:18.055951 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af356c3-60f7-45a8-8d5a-3cbb6c64baf6 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:18:18.063876 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1465/2920] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:18:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 18:18:18.527669 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc8f6a-82bc-419d-82e8-20195efcf2fb service cinder] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:18:18.537696 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bc8f6a-82bc-419d-82e8-20195efcf2fb service cinder] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:18:18.541395 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0952c4-13e2-490c-b0fe-77b940f111c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:18:18.542042 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0952c4-13e2-490c-b0fe-77b940f111c4 service neutron] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:18:18.542333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0952c4-13e2-490c-b0fe-77b940f111c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:18:18.542617 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0952c4-13e2-490c-b0fe-77b940f111c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:18:18.553635 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0952c4-13e2-490c-b0fe-77b940f111c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:18:18.555211 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0952c4-13e2-490c-b0fe-77b940f111c4 service neutron] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:18:18.567334 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1456/2921] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:18:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:18:20.235286 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af356c3-60f7-45a8-8d5a-3cbb6c64baf6 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:18:20.245116 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af356c3-60f7-45a8-8d5a-3cbb6c64baf6 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:18:20.248848 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4e818-2b38-4746-9d5b-bc0ebe237f19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:18:20.249514 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4e818-2b38-4746-9d5b-bc0ebe237f19 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:18:20.249799 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4e818-2b38-4746-9d5b-bc0ebe237f19 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:18:20.250082 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb4e818-2b38-4746-9d5b-bc0ebe237f19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:18:20.259367 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4e818-2b38-4746-9d5b-bc0ebe237f19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:18:20.260850 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb4e818-2b38-4746-9d5b-bc0ebe237f19 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:18:20.270038 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1466/2922] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:18:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:18:20.882483 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0952c4-13e2-490c-b0fe-77b940f111c4 service neutron] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:18:20.896779 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0952c4-13e2-490c-b0fe-77b940f111c4 service neutron] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:18:20.916958 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ce49b-bdc5-41a6-bf99-1d1ec02bffa3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:18:20.918446 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ce49b-bdc5-41a6-bf99-1d1ec02bffa3 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:18:20.918959 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ce49b-bdc5-41a6-bf99-1d1ec02bffa3 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:18:20.919470 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1ce49b-bdc5-41a6-bf99-1d1ec02bffa3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:18:20.922647 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-1a1ce49b-bdc5-41a6-bf99-1d1ec02bffa3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:18:20.937868 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ce49b-bdc5-41a6-bf99-1d1ec02bffa3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:18:20.939762 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1ce49b-bdc5-41a6-bf99-1d1ec02bffa3 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:18:20.952163 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1457/2923] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:18:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:18:30.265749 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=15 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:18:30.947578 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=18 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:18:36.217104 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4e818-2b38-4746-9d5b-bc0ebe237f19 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:18:36.225696 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb4e818-2b38-4746-9d5b-bc0ebe237f19 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:18:36.229267 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5ae3d-4b8a-4787-9115-f34c1336f176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:18:36.229903 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5ae3d-4b8a-4787-9115-f34c1336f176 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:18:36.230189 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5ae3d-4b8a-4787-9115-f34c1336f176 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:18:36.230468 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5ae3d-4b8a-4787-9115-f34c1336f176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:18:36.239739 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5ae3d-4b8a-4787-9115-f34c1336f176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:18:36.241185 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a5ae3d-4b8a-4787-9115-f34c1336f176 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:18:36.250424 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1467/2924] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:18:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:18:37.998662 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ce49b-bdc5-41a6-bf99-1d1ec02bffa3 service nova] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:18:38.010625 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1ce49b-bdc5-41a6-bf99-1d1ec02bffa3 service nova] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:18:38.016501 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de55eb-4d27-4853-b40f-8bb36e5c6808 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6423f54152064b909d03c0908d636993', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:18:38.017259 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de55eb-4d27-4853-b40f-8bb36e5c6808 service nova] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:18:38.017621 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de55eb-4d27-4853-b40f-8bb36e5c6808 service nova] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:18:38.017974 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de55eb-4d27-4853-b40f-8bb36e5c6808 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:18:38.028043 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de55eb-4d27-4853-b40f-8bb36e5c6808 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:18:38.029746 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de55eb-4d27-4853-b40f-8bb36e5c6808 service nova] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:18:38.041866 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1458/2925] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:18:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:18:46.246311 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=6 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:18:48.036985 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=6 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:20:41.269649 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5ae3d-4b8a-4787-9115-f34c1336f176 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:20:41.290165 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5ae3d-4b8a-4787-9115-f34c1336f176 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:20:41.293861 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c4138-00bb-4b08-a2e4-ab39c2e79458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a5adec9924444e29b8964f7969452592', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:20:41.294503 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c4138-00bb-4b08-a2e4-ab39c2e79458 service neutron] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:20:41.294785 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c4138-00bb-4b08-a2e4-ab39c2e79458 service neutron] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:20:41.295065 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c4138-00bb-4b08-a2e4-ab39c2e79458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:20:41.375710 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c4138-00bb-4b08-a2e4-ab39c2e79458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:20:41.377169 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c4138-00bb-4b08-a2e4-ab39c2e79458 service neutron] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:20:41.387014 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1468/2926] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:20:44.155690 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de55eb-4d27-4853-b40f-8bb36e5c6808 service nova] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:20:44.157310 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-c2de55eb-4d27-4853-b40f-8bb36e5c6808 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:20:44.260447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de55eb-4d27-4853-b40f-8bb36e5c6808 service nova] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:20:44.267257 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77c13b-8442-4c93-bb25-1943ebd77733 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:20:44.268095 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c13b-8442-4c93-bb25-1943ebd77733 service placement] REQUEST_METHOD: `GET` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:20:44.268385 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c13b-8442-4c93-bb25-1943ebd77733 service placement] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:20:44.268662 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c13b-8442-4c93-bb25-1943ebd77733 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:20:44.270648 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-fd77c13b-8442-4c93-bb25-1943ebd77733 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:20:44.284616 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77c13b-8442-4c93-bb25-1943ebd77733 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:20:44.286237 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77c13b-8442-4c93-bb25-1943ebd77733 service placement] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:20:44.300738 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1459/2927] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:20:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:20:47.246671 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c4138-00bb-4b08-a2e4-ab39c2e79458 service neutron] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:20:47.258232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c4138-00bb-4b08-a2e4-ab39c2e79458 service neutron] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:20:47.263964 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b4294-3579-4bb2-b673-4ce6bcf0c44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:20:47.264887 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b4294-3579-4bb2-b673-4ce6bcf0c44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:20:47.265502 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b4294-3579-4bb2-b673-4ce6bcf0c44e admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:20:47.265625 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b4294-3579-4bb2-b673-4ce6bcf0c44e admin admin] PATH_INFO: `/v3/users/15d9d47d160b4595a16bf23eaa509a08` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:20:47.266815 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b4294-3579-4bb2-b673-4ce6bcf0c44e admin admin] RBAC: Authorizing `identity:delete_user(user_id=15d9d47d160b4595a16bf23eaa509a08)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:20:47.269021 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64b4294-3579-4bb2-b673-4ce6bcf0c44e admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:20:47.315162 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b64b4294-3579-4bb2-b673-4ce6bcf0c44e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15d9d47d160b4595a16bf23eaa509a08'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.327053 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b64b4294-3579-4bb2-b673-4ce6bcf0c44e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15d9d47d160b4595a16bf23eaa509a08'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.334359 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b64b4294-3579-4bb2-b673-4ce6bcf0c44e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15d9d47d160b4595a16bf23eaa509a08'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.334967 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b64b4294-3579-4bb2-b673-4ce6bcf0c44e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15d9d47d160b4595a16bf23eaa509a08'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.342296 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b64b4294-3579-4bb2-b673-4ce6bcf0c44e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15d9d47d160b4595a16bf23eaa509a08'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.348277 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1469/2928] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:20:47 2022] DELETE /identity/v3/users/15d9d47d160b4595a16bf23eaa509a08 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:20:47.524447 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77c13b-8442-4c93-bb25-1943ebd77733 service placement] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:20:47.536653 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77c13b-8442-4c93-bb25-1943ebd77733 service placement] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:20:47.592412 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:20:47.593262 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:20:47.593543 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:20:47.593815 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] PATH_INFO: `/v3/projects/3a189153b8f44520a5bac2c4bcedc013` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:20:47.594983 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a189153b8f44520a5bac2c4bcedc013)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:20:47.596897 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:20:47.642354 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a189153b8f44520a5bac2c4bcedc013', 'request_id': 'req-ff43c288-a609-433a-a012-abb042e08abf'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.649679 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a189153b8f44520a5bac2c4bcedc013', 'request_id': 'req-ff43c288-a609-433a-a012-abb042e08abf'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.655103 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a189153b8f44520a5bac2c4bcedc013', 'request_id': 'req-ff43c288-a609-433a-a012-abb042e08abf'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.665750 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] The token cache is being invalidate because project 3a189153b8f44520a5bac2c4bcedc013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:20:47.666009 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.666643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.668207 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1460/2929] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:20:47 2022] DELETE /identity/v3/projects/3a189153b8f44520a5bac2c4bcedc013 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:20:47.672379 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b4294-3579-4bb2-b673-4ce6bcf0c44e admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:20:47.673807 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b64b4294-3579-4bb2-b673-4ce6bcf0c44e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:20:47.687056 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64b4294-3579-4bb2-b673-4ce6bcf0c44e admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:20:47.719263 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:20:47.745521 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d6ea1-ef9b-4ff7-937c-58412375ae75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:20:47.746698 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d6ea1-ef9b-4ff7-937c-58412375ae75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:20:47.746983 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d6ea1-ef9b-4ff7-937c-58412375ae75 admin admin] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:20:47.747258 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d6ea1-ef9b-4ff7-937c-58412375ae75 admin admin] PATH_INFO: `/v3/users/8d639a61029d49a78164037e89adfc41` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:20:47.748511 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d6ea1-ef9b-4ff7-937c-58412375ae75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d639a61029d49a78164037e89adfc41)` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:20:47.750450 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d6ea1-ef9b-4ff7-937c-58412375ae75 admin admin] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:20:47.794355 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b73d6ea1-ef9b-4ff7-937c-58412375ae75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d639a61029d49a78164037e89adfc41'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.806456 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b73d6ea1-ef9b-4ff7-937c-58412375ae75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d639a61029d49a78164037e89adfc41'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.815945 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b73d6ea1-ef9b-4ff7-937c-58412375ae75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d639a61029d49a78164037e89adfc41'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.816579 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b73d6ea1-ef9b-4ff7-937c-58412375ae75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d639a61029d49a78164037e89adfc41'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.824844 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.notifications [None req-b73d6ea1-ef9b-4ff7-937c-58412375ae75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d639a61029d49a78164037e89adfc41'} {{(pid=91926) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:47.831017 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1470/2930] 10.222.0.64 () {58 vars in 1223 bytes} [Thu Jul 14 18:20:47 2022] DELETE /identity/v3/users/8d639a61029d49a78164037e89adfc41 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:20:48.013572 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] Authenticating user token {{(pid=91925) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:20:48.014992 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.fernet_utils [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91925) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:20:48.027405 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff43c288-a609-433a-a012-abb042e08abf admin admin] Validating token access rules against request {{(pid=91925) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:20:48.055115 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:20:48.087033 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d77dedfd6bd435d94f994cfcde5100a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '29b8bf26bc324367a4fe27e90fc2d6df', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91925) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:20:48.087892 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:20:48.088203 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:20:48.088589 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9 admin admin] PATH_INFO: `/v3/projects/8ff43d2a08634ce4aa7a214ea2951f98` {{(pid=91925) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:20:48.089843 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ff43d2a08634ce4aa7a214ea2951f98)` {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:20:48.091755 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9 admin admin] RBAC: Authorization granted {{(pid=91925) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:20:48.140665 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ff43d2a08634ce4aa7a214ea2951f98', 'request_id': 'req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:48.147965 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ff43d2a08634ce4aa7a214ea2951f98', 'request_id': 'req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:48.153643 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ff43d2a08634ce4aa7a214ea2951f98', 'request_id': 'req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9'} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:48.164408 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9 admin admin] The token cache is being invalidate because project 8ff43d2a08634ce4aa7a214ea2951f98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=91925) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 18:20:48.164696 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:48.165333 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG keystone.notifications [None req-6cdb6ca7-462a-47f6-a57a-c4ba3dfcfcb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=91925) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 18:20:48.167423 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: [pid: 91925|app: 0|req: 1461/2931] 10.222.0.64 () {58 vars in 1232 bytes} [Thu Jul 14 18:20:48 2022] DELETE /identity/v3/projects/8ff43d2a08634ce4aa7a214ea2951f98 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 18:20:57.827400 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=37,keystone:DELETE=10,keystone:INSERT=1 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:20:58.161388 dv-cinder-849890-1-iscsi devstack@keystone.service[91925]: DEBUG dbcounter [-] [91925] Writing DB stats keystone:SELECT=38,keystone:DELETE=7,keystone:INSERT=1 {{(pid=91925) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 18:21:19.170811 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d6ea1-ef9b-4ff7-937c-58412375ae75 admin admin] Authenticating user token {{(pid=91926) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 18:21:19.172232 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-b73d6ea1-ef9b-4ff7-937c-58412375ae75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:21:19.250092 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d6ea1-ef9b-4ff7-937c-58412375ae75 admin admin] Validating token access rules against request {{(pid=91926) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 18:21:19.307993 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45f8bd-4d2a-4a47-9d5c-88c41d9b8c7e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '619e3bec841343c5ad144f1d4e4d436d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'de62d2fb17f846f2bce5bef108353fd8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=91926) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 18:21:19.308649 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45f8bd-4d2a-4a47-9d5c-88c41d9b8c7e service placement] REQUEST_METHOD: `GET` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 18:21:19.308936 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45f8bd-4d2a-4a47-9d5c-88c41d9b8c7e service placement] SCRIPT_NAME: `/identity` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 18:21:19.309219 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45f8bd-4d2a-4a47-9d5c-88c41d9b8c7e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=91926) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 18:21:19.311016 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.fernet_utils [None req-ca45f8bd-4d2a-4a47-9d5c-88c41d9b8c7e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=91926) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 18:21:19.322260 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45f8bd-4d2a-4a47-9d5c-88c41d9b8c7e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 18:21:19.323843 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45f8bd-4d2a-4a47-9d5c-88c41d9b8c7e service placement] RBAC: Authorization granted {{(pid=91926) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 18:21:19.333751 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: [pid: 91926|app: 0|req: 1471/2932] 10.222.0.64 () {60 vars in 1365 bytes} [Thu Jul 14 18:21:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 18:21:29.329544 dv-cinder-849890-1-iscsi devstack@keystone.service[91926]: DEBUG dbcounter [-] [91926] Writing DB stats keystone:SELECT=62 {{(pid=91926) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}